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SUMMARY 

The results of the 1982-83 quality control study of the Pell Grant program are 
presented and. discus^d in this volume/ In general, the findings are indicative of 
improvement but also demonstrate the need for continuing management attention. 

Total dollar ^rror is estimated to be $256 per recipient, or $650 milUon, 
representing 27 percent of the total program expenditure for this program with 
2.5 million recipients.- About 6 out of lO.reci^nts received incorrect awards. 
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Other key findings ^include: 

• On average. Pell Grant "recipients in 1982-83 were granted $129, or 13 
percent, more than they should have been. This , compares with an average 
$170 net error in 1980^81 and represents a 2^ percent decrease in average 
net error over the two year period. \^ 

• The net effect of the '$129 average oyerpSyment for the program's 2.53 
million recipients in 1982-83 was an estimated $326 million overpaid by the 
program. Overawards exceeded the absolute value-^f underaWards by over 
3tol. . 

• More than 62 percent of the students had errors in award over $2 and more 
than ^2 percent had errors in award over $iOO. ,^n estimated 11.7 percent 
of Pell Grant recipients (300,000 students) sKoiild not have been given a 
grant. This represents a 36r percent decrease from the estimated ^70,000 
ineligibles in 1980-81 who represented 20 percent of that year's recipients. , 

• 3ust under ^0 percent of the Pell Grant recipients made mistakes in the 
data submitted on their application, forms that affected their awards. 
These student-generated errors resulted in a net $86 overaward per 
recipient in. the program or $217 million ove^^ail. Student-generated 
overawards exceeded the absolute value of undera^rds by almost 5 to 1. 

• • Mistakes by institutions caused errors for 33 percent of all recipients. 

institution-generated errors resulted in a net $99 million in overpayments 
• to Pell Grant recipients, .with overawafds exceeding^the absolute value of 
unj^erawards by about 2 to T. « v * 

• Lack of a ?jgfancial Aid Transcript was the major cdhtributor to institu- 
tional errcfTFor analytic purposes, the lack of a Financial Aid Transcripts^:' 
was combined ^ with the two other procedural errors: lack of a signed 
Stjitement of Educational Purpose and lack of a valid (original) Student Aid 
Report. Disregarding these procedural errors, institutions^ctually under- 

. paid Pell Grant recipients a total of $13 million dollars with underawards 
being about 13 percent greater than overawards. 
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The 1982-83 Pell Grant Program experienced an estimated $217. million in 
net student error, or.$86 t>er recipj^nt. 

Dependency'status 'error wa^ the largest single form of student error, 
comprising approximately $6* i\iIliorfin payment consequences- " : 

The top six forms of student error were not directly^ verifiable through 
1982-83 validation procedures, ' ■ 



Tlie vast majority bf institutions collected the required verifying* 
documentation for their students who were "flagged" for validation by ED. 
The Federal tax return was the predominant form of documentation. 

Validated students were more likely to revise their Adjusted Gross Income 
(AGI) and Federal taxes paid initial application' data than were non- 
validated students. Further^^ those selected for validation and making 
revisions were l7iore likely to raise their AGI and reduce their taxes^ paid- 
revisions that tend to lower eligibility— than were those not selected for 
validation. ' 

^- ' ■ ■ ■ ' ' 

Among students y/ho made changes to their application, validated Students 
were much more- likely 'to show an increased SAI, leading to a decrease in 
award. Corrections behavior in this direction was very -infrequent among 
nonvalidated students. 

M.ost institutions believed that they were unduly burdened by the expanded 
nature of the 1982-83 validation process. The reasons cited most often 
were delays, extra- work, or confusion due to the late arrival of the 
Validation Handbook; difficulty in. verifying Social Security benefits and 
Veterans Administration b^efits; and difficulty irf obtaining 
documentation from students. . . 
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4 CHAPTER 1 
INTRODUCTION 

This volume is one of a series that documents Stage. Three of the Pell Grant 
Quality Control Study. The other two volumes coi%ern,the methods and procedures 
used and recommeflndations for actions to correct the *oblems found. 

In September, 1980, the Office of«tudpnt Financial Assistance (OSFA) of the 
U.S. Department of Education (ED) contracted with Advanced Technology, Inc., of 
McLean and Reston, Virginia^ to conduct a three-year study to assess the accuracy and 
reliability of the Basic Educational Opportunity Grant (BEOG) Program and recom- 
mend administrative changes to improve it. Westat, Inc., of Rockville, Maryland, has 
served as a subcontractor to Advanced Technology throughout the study. In. 1981, the 
name of the program was changed to the Pell Grant Program. 

During Stage One of the study—the first year of the contract (1980-81)— 
Advanced Technology and Westat examined a national sample of Pell Grant recipients 
to determine eligibility and award calculation error. Westat drew the national sample 
of 1^,500 fefcipients and interviewed them and their parents about their eligibility and 
financial situation, examining documentation at the same time. Advanced Technology 
hired staff to visit the institutions attended by these students, examine the records on 
.the students" and a:ny supporting documents on file, and interview the financial aid 
administrators. This' data collection procedure was also followed in Stage Three— the 
third year of the contract (1982-83)— and is described below. 

The "Stage One stu|^" determined program-wide rates of discrepancy between 
actual awards and the aWjaircis that should have been made according to program rules 
and the documents Examined, and attributed these discrepancies to institutions, 
recipients, or their parents, and application processors. On the basis of these 
discrepancy rates, the study also identified error-prone groups of recipients. Finally, 
Stage One suggested feasible corrective management activities tp reduce error rates 
for every area in which error rates were excessive. 



During Stage Two (October, 1981 to December, 1982) Advanced Technology 
^jegan the design of a quality control system for the Pell Grant program and made 
some error analyses and corrective action recommendations for specific features of 
related student aid programs. The Department of Education began installing correc- 
tive measures, including a requirement for greatly increased validation of Pell Grant 
applicants on a limited number of application items, rather than a small sample on 
more items. In 1983, the quality control system design component became a separate 
project. 

Stage Threfe of the study (1982-83) has essentially been a repUcation of Stage 
One, with the objective of determining changes in program error over time, especially 
changes potentially brought about by the extended validation requirement. As in Stage 
One, Westat developed a.sampling procedure and interviewed the sampled parents and 
students, and Advanced Technology visited institutions to examine documents in 
student files and interview financial aid administrators. During Stage Three Advanced 
Technology made a preliminary visit to each of 317 institutions to draw the sample of 
'f,i09 students on site (during Stage One Westat had drawn the sample at its home 
office from lists supplied by the institutions) and to gather data for an assessment of 
compliance with the new validation requireifjent. 

1.1 THE PELL GRANT PROGRAM 

In the 1982-83 academic year, the Pell Grant program distributed over $2.'f 
billion to over 2.5 million students who attended over 6,000 institutions of post- 
secondary education. These Pell Grants are designed to serve as the base of Federal 
aid for students from lower-income families. The major purpose of the Feideral role in 
student financial aid is to equalize access to postsecondary education. 

The Pell Grant program has certain unique characteristics. First, eligibility 
standards are uniform across all schools, and students. Second, the Pell Grant is 
portable in that it can be used to finance the cost of education at any eligible school 
selected by the student. Third, Pell Grant delivery involves numerous actors. For 
example, a student can apply for a Pell Ctftnt through any of four different application 
processors. Fourth, it is a Federal program which relies heavily on individual 
institutions for program administration. ' 



The program has grown steadily since its enactment on June 23, 1972 as an 
amendment to the Higher Education Act of 1965. In 1973-7^ the program distributed^- 
$5a million to 185,000 students. Currently, about 2.7 million students are receiving 
almost $2.5 billion. ^ >• '/ ' 

1.2 QUALITY CONTROL- IN THE OFFICE OF STUDENT FINANQAL ASSISTANCE 

The Office of Student Financial Assistance (OSFA) is attempting to have 
inctividOals responsible for the various processes and activities which make up the 
student aid delivery system assist in quality control. Controlling quality must- be a 
responsibility of these operating groups, with OSFA taking the role of assuring that : 
these groups have procedure^ and processes in place which will control the> quality of 
their work and activities. This distinction between quality control and quality 
assurance is a key concept in QSFA's quality improvement program. 

The quality improvement Iprogram has' many facets: 

Pell Grant Quality Control studiesj^ 

Development of corrective action* for selected internal "targets of oDDor- 
tunity" . \ 

'Vendor quality control requirements for all OSFA procurements 

Validation selection targeting using error-prone modeling 

' Quality ^control regulations for Campus-Based programs currently being 
developed by OSFA 

' 'Pilot quality control study of Campus-Based programs and GSL certifica- 
;-' tion > 

'■*' 

^ . Field testing of Pell Grant Application forms - 

Computerized edits as part of the application processing system 
f ' Student financial aid training programs. 

1.3 PURPOSE AND OBJECTIVES OF THE PELL GRANT QUALITY CONTROL 
/ STUDIES , 

This study of the 1982-83 program year is the third Pell Grant quality control 
study. The earlier studies focused on program years^ 1978-79 and 1980-81. All three 



studies have the same general purpose and objectives. For the current 1982-83 study 
the specific objectives are: 

Estimate program-wide error rates - - 

Identify probable causes of error 
Develop and analyze alternative corrective actions ' 
Evaluate the effectiveness of institutional validation 
Compare the 1982-83 findings with the 1980-81 and 1978-79 findings. 

The results of this effort are reported in a four-volume report of which this is 
Volume 1. The four volumes are: 

Volume 1: Findings 
Volume 2: Corrective Actions 
Volume 3: Methods and Procedures 
Executive Summary. 

Following this introductory chapter, chapter 2 of this volume presents the 
program-wide estimates of error. Identification of probable causes of error is 
discussed in Chapter 3 for institutions and. Chapter for students. Evaluation of the 
effectiveness of validation is discussed in Chapter 5, while Chapter 6 compares 
current findings with previous studies. Methodology is briefly discussed throughout 
this volume and more extensively in Volume 3. Chapter 7 discusses methodological 
issues which might affect the validity of the study. ' , 

l.'^ GENERAL STUDY DESIGN . 

The general approach to this quality control study is to compare the "reported ' 
values" and "best values" for variables used in the application and award determination 
processes for a nationally representative sample of Pell Grant recipients. Reported 
values are those supplied by parents and students on the application form or those 
institutionally provided items utilized by the financial aid and other offices. 

"Best values" are derived from information collected for all recipient cases using 
a multi-instrument data collection process. These instruments include: 
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• 3,576 completed interviews with the student recipients 

• 3,060 completed interviews with the parents of the sampled recipients 

• 3,786 completed student record abstracts of student files at the institu- 
tions * ^ " . . 

*,007 IRS certified copies of 10^0 tax forms for parents, students and 
spouses 

. • 3^9 statements from local tax assessors regarding home values for a 25 
percent subsample of the cases 

• • ^ 365 statements from banks, savings and loan associations, ' ^nd other 
, financial institutions documenting balances for parents and recipients with 

^ large holdings (over $^,000) of highly liquid assets 

• Interviews with the financial aid administrators at the 3 17 schools attended 
by the sampled recipients 

• ' Complete application histories for each sampled recipient, from the Pell 

Grant central processor 

, • Information from other ED information systems. 

The^sample of ^,109. consisted of ^,082 recipients at 317 institutions and 27 
students attending institutions which do not administer Pell Grants (Alternate Dis- 
bursement System schools). The sample was designed to be representative of all Pefl 
Grant recipients. Detailed sampling issues are discussed in Volume 3, Methodology/ 

p * ' 

1.5 DETAILED DEFINITION OF ERROR 

The proper calculation and disbursement of a PelLGrant require many pieces of 
data from many sources. Therefore, error measurement requires the identification 
and enumeration of these elements as well as their combinations. Figure 1-1 shows 
the hierarchy of data elements used in Pell Grant award determination. The left-hand 
column represents the most detailed level of data items. Going toward the right 
represents higher-level combinations of thesfs elements. 

Error takes on one of two forms: fir?t, the value for any variable or data 
element could be wrong. Second, transcriptions or calculations using the basic data 
elements could be incorrect. There are two ways to measure the occurrence or 
severity of these errors. One is simply to count the number of times an error occurs 




ITEMS 



COMPONENTS 



SOURCES 



Adjijsted Gross Income 
Taxes Paid - 

Father's/ Applicant's Income 
Mother's/ Spouse's Income 
AFDC 

Social Security Benefits 
Other Income/Benefits 
Household Size ' . 
Number in Postsecondary 

Education 
Medical/Dental Expenses 
Elementary/Secondary School : 

Tuition 
Cash/Savings/Checking 
Home Value 
Home Debt 

Real Estate/Investment Value 
Real Estate/Investment Debt 
Business/Farm Value 
Business/Farm Debt 
Dependent Student's (and Spouse's) 

Net Income 
Dependent Student's (and Spouse's) " 

Net Assets 
Expected VA Educational Benefits 
Expected Social Security Educational 

Benefits 
Parent's Marital Status 
Student's Marital Status 
Support by Parents, 1981 
Support by Parents, 1982 
Claimed by Parents, 1981 
Claimed by Parents, 1982 
Lived with Parents, 1981 
Lived with Parents, 1982 

Missing Student Aid Report 
Invalid Stiident Aid Report 
No Statement of Educational 

Purpose 
No Financial Aid Transcript 
Insufficient Program Length 
Ineligible Program 
Has BA Degree 
Not Citizen 
Not Home School 
Loan or Grant Repayment Default 
Less than Half-Time 
Unsatisfactory Academic Progress 
Cost of Attendance 
Enrollment Status 
Calculation/Accounting 
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Error 



[Nontaxable 
Income 
[ Error 



Net 

Worth 

Error 



Dependency 

Status 

Error 



Student 
Error . 



Eligibility 
Error 



Disbursement Error 



Institutional 
Error 
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ERROR COMPONENTS, ITEMS AND COMBINATIONS 



and express thi$ as a percentage error rate. The. second measure is to determine the 
change in payment which' would result from correcting the incorrect data element or 



calculation. 



The payment consequence error measure is the difference between two payment 

amounts. The first payment amount is what the student woqld haye received if error 

• in the' particular variable(s) was not corrected. This is an award based on reported 

values. The second payment amount is what the student would have received if 6rror 

in the particular varidble(s) was corrected. This is an award based on best values. 

. - It ^ 

This definition of payment consequence can be used for error in one variable, a set of 

relarted variables (e.g., dependency status questions or tax form items), all student 
provided data, all institutionally provided data, or all data elements jointly. The 
values of -variables or data elements not being considered in the error measure are 
generally set at their application values except for overall student and overall 
institution errors. Appendix A provides more detailed algebraic representations for 
the various error composites. 

1.6 STUDY ASSUMPTIONS AND LIMITATIONS 

• The data and estimates provided in the following chapters are based on certain 
assumptions and are thus subject to certain limitations. 

1.6.1 Confirmatory Nature of the Study 

The study approach attempts to confirm the values reported by recipients and 
their parents. The study does not attempt to investigate independently, the financial 
status of recipients and their parents. Thus, errors of commission, people reporting 
the wrong value for an income source, are likely to be uncovered by the study. Errors 
of omission, failure to report any value for certain income sources, are not as.likely to 
be uncovered by the study protocol. 

i ^- ^" 

As a result, the estimates provided here will understate error to the extent that 
errors of omission are not uncovered by the confirmatory approach. 
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1.6.2 Presumption of No Error V 

The general approach to the study is to assume that the application value for a 
data element is correct unless there are study data to indicate the contrary. Thus, any 
data- elemetit that the recipient or parent was unable to document was assumed to 
have been correct on the application and no error was noted. To the extent that 
sampled recipients fail to respond to ^certain questions when the likelihood for error is 
high,, this presumption of no error will result in estimates which understate the true 
level of error. " - , 

L6.3 Timing of Data Collection and Self-Correction 

Institutional data collection visits occurred as late in the academic year as 
practical, given the reporting needs of this study. However, it is possible that the 
estimates presented here overstate institutional error because they do not reflect the 
self-correctibns generated by the schools in response to program year close-out and 
reconciliation. We do not feel that this is a serious problem becaase of our experience 
with the 1980-81 Quality Control Study. In a special analysis^ we utilized 1980-81 end- 
of-year, reconciled values and found little self-correction to have occurred after the 
institutional site visit. / 

L6A Nonresponse Bias 

' Any survey is subject to the problem of nonresponse bias. This may be caused hy 
people who knowingly misreported data on their applications and refused to cooperate 
with the interviews. However, our overall student response rate of about- 90 percent 
indicates that the problem is rather well contaihed. In addition, we have assessed the 
demographic differences between respondents and nonrespondents and found them to 
be minimal. Finally, we assessed the sensitivity of the results to various assumptions 
concerning nonrespond^ts. This analysis is discussed in Chapter 7. 

L6,5 Experimental Bias 

The field work and data collection employed in this study may have caused the 
sampled recipients, their parents, and their institutions to alter their behavior. The 
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nature of this, behavior modification wouid be to iower the ievel of error measured by 
this study. 

To assess this potential downward bias, two control groups were constructed. 
The results from these two special groups indicate that experimental bias does not 
cause serious downward bias in the estimates." This also is discussed in Chapter 7. 
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/ CHAPTER 2 ^ 

_ . OyERVffiW OF PROGRAM- WIDE ERROR ^. 

in this cfiapter, estimates of Pell Grant program-wide -err^r fpr 1982-83 are 
presented. Highlights of ^he findings prese'nted in.this chapter include: 

, • , On average, Pell Grant, recipients in 1982-83 wer? granted $129, or 13 
percent, more than they should have been. This compares with an average 
5170 net error in 1980-81 and represents a 2'f percent decrease in average 
net error over the two year period. 

• The net effect of the $129 average overpayment for the program's 2.53 
million recipients in 1982-83 was an estimated $326 mUlion overpaid by the 
program. Overawards exceeded the absolute value of underawards bv over 
3 to 1. " • ' 

• More than 62 percent/of the students had errors in award over $2 and more 
than *2 percent had errors in award aver $100. An estimated 11.7 percent 
of Pell Grant recipients (300,000 students) should not have been given a" 
grant. This represents a 36 percent decrease from the estimated 'f7G,000 
ineligibles in 1980-81 who represented 20 percent of that year's r,ecipients. 

• 3ust under W percent of .the Pell Grant recipients made mistakes in the 
data submitted on their application forms that affected their awards. 
These student-generated errors resulted in a net $86 overaward per 

• recipient in the program or $217 mUlion overall. Student-generated 
overawards exceeded the absolute value of underawards by almost 5 to 1. 

• Mistakes by institutions caused errors for 33 percent of all recipients. 
Institution-generated errors resulted in a net $99 mUllon in overpayments 
to Pell Grant recipients, with overawards exceeding the absolute value of 
underawards by about 2 to 1. 

'• Lack of a Financial Aid Transcript was the major contributor- to institu- 
tional error. For analytic purposes, the lack of a Financial Aid Transcript 
was combined with the two other procedural errors: lack of a signed 
Matement of Educational Purpose and lack of a valid- (original) Student Aid 
Report. Disregarding, these procedural errors, institutions actually under- 
paid Pell Grant recipients a total of $13 mUlion with underawards being ^ 
about 13 percent greater than overawards. 
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2.1 OVERVIEW OF GASE/TOTALi ERROR 

'■'!■'■■'. 

y The dollar amounts and rate^ of error were defined ih several ways. In this 
^-^tion the definitions of the various ern?r types are presented. This is followed by 
text and tables showing the amounts and rates of error found for 1982-83. 

2. 1.1 "Types of Error 

... - 1 . . , 

The error figures presented in this report generally fall into one of* four 
categories: absolute error, n^st error, overaward- error, and underaward error. Stu- 
dents who received less than was determined to be the correct amount are said to have 
been "underawarded." Similarly, students who received more than was determined to 
be the correct amount are said to have been "overawarded." 

The sum of the dollar error for all "underawarded" recipients is called the 
"underaward error, while the sum for all "overawarded" recipients is* called the 
overaward error. Subtracting the value of underaward error from overaward error 
yields net error.. This amount represents the estimated payments in error made to 
students. For most sources of error, net error is a positive number, representing 
excess cost to the program, because the value of overawards is usually greater than 
the absolute value of underawards. If underawards exceed overawards, net error is a 
negative number. Adding the absolute value of underaward error to overaward error 
yields absolute error. .. This amount represents the sum of the estimated incorrect 
payments ma* to students, whether that incorrect payment be an excess or a 
shortfall. 

An attempt also was made to distinguish among institutionally generated error, 
student-generated error, total error, and case error. Institutional error is defined as 
the difference between a recipient's actual award and the amount the school should 
have given the stjiident based on the Student Aid Index (SAI) the school had in its files. 
This reflects the school's incorrect determination of the student's eligibility to 
participate in the program as well as errors in determining or using the correct cost of 
attendance and/or enrollmgpt status. Student error is "defined as the difference 
between the correct award the student should have received and the award calculated 
using the application data submitted by the student and the correct cost of attendance 



and enrollment status. This reflects the marginal effect on the recipient's award 
effect caused by errors in the student's application data. 

Adding institution error and student error yields total error. For' some 
recipients, awards are affected by both student and institutional errors. In many 
. cases, there are compensating errors, for total error,' any compensatory ef facts , 
student and ^institutional error are' ignored. Case error includes the effects of 
compensatory student and institutional errors. It is defined to be the difference 
between the 'award the student, received and. the award the student should have 
received using all of the best data available. When the term "error" is used alone in 
this report, it usually refers to case error. 

Each of the types of error defined here can be presented in terms of rate of 
error and amount of error. Rate of error is simply the percentage of cases with error. 
This percentage is based on the sample but is also the best estimate for the population 
of Pell recipiehts. Amount of error can be presented in several different ways. Two 
types of average error are used throughout this yolume. Mean error per recipient is 
the total dollar amount of error divided by the total number of recipients. It provides 
an average figure for all recipients, with recipiems with, zero^rror included in the 
number, of, recipients. Mean error per recipiait with error is the total dollar amount of ^ 
error divided by the number of recipients with error. It provides an average figure 
only for those recipients with error, excluding all ^^^ecipients with zero error. While 
both measures of average error are derived from the sample, each is a good estimate 
of error in the population of Pell recipients. v 



The final error figure used is the program-wide estimate, presented in milfions of 
dollars. It is derived by multiplying the jjp^an error per recipient by the estimated 
number of Pell recipients, 2.53 million. The program-wide estimate also is- presented 
as a percentage of the total value'sf all 'Pell awards in 1982-83, approximately $2.f 
billion. / 



^ 

2.1.2 Measures of Case/Total Error 



Table 2-1 presents a summary of the amounts and rates of error estimated in the 
Pell Grant program for 1982-83. An estimated 62.7 percent of all recipients had 
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TABLE 2-1 

A SUMMARY OF AMOUNTS AND RATES OF ERROR 



7 



Error 



Institutional 
Student 

Toul 

r 

Case 



ABSOLUTE ERROR 



Program-Wide Estimate ^ 
($MlUk»u) A^IL^H 



321 
328 
649 
603 



13 
14 
27 
23 



Mean 
Error per Cases 

Recipient . W/Error^ 
($) (%) 



Mean 
Error per 
Recipient 
W/Error 
($) 



NET ERROR 



Error 



Program-Wide Estimate 
($MUllons) A^^?^ 



Mean 
Error per Cases 

Recipient W/Ernir^ 
(S) (%) 



Mean 
Error per 
Recipient 
W/Error 
($) 



J27 


33.3 


- 379 


Institutional 


99 


% 


39 


33.3 


117 


129 


39.4 . 


328 


Studeiht 


217 


9 


86 


39.4 


217 


236 


62.7 


408 ' 


Toul 


316 




123 


62.7 


199 


239 


62.7 


381 


Case 


326 


13 


129 


62.7 


203 



t 



Error 


Program-Wide Estimate 
($l^ons) Ai^dli)^ 


Mean 

Error per 
Recipient 
(I) 


Cases 
W/Error^ 
(%) 


Mean 

Error per 
Recipient 
W/Error . 


Error 


UNDERAWARD ERROR 
Mean 

Program-Wide Estimate Error per 
($ Millions) Ai^iSii^ ^""^^ 


Cases 
W/Errorl» 
(%) 


Mean 
Error per 
Recipient 
W/Error 
($) 


Institutional 


210 


9 


83 


13.7 


328 


Institutional 


-111 


3 -44 


17.8 


-247 


Student 


272 


11 


108 


' 30.6 


331 


Student 


-33 


2 -22 


8.8 


-249 


Total 


482 


; 20 . 


190 


41.3 


439 


Total 


-166 


7 166 


21.2 


-309 


Case 


463 


19 


184 


41.3 


444 


Case 


-139 


6 -33 


21.2 


-239 



* Amount oi Pell awards is $2.4 billion lor 1982-83. 
Error is defined as a discrepancy of plus or minus $2 from the best award. 
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'awards in error by at least $2. [ Extending this tolerance range to $100 drops the 
percentage of recipients in error to ^2.^ pcJrc^t. However, the p'rogram-^wide 
estimates oLerror, both in an absolute and net sense,, differ only slightly between the 
$2 and $100 tolerance ranges, l^iig is shbwn in Table 2-2. The estimated absolute 
eYror with a $2 tolerance is $605 millioh, while All^ith a $100 tolerance it is $577 
million. Similarly, estimated net error drops by only 1 percent with the expanded 
tolerance range. ' . \^ ' ' 

\ On. averagej Pell Gi^ant recipients in 1982-83 received $129 too much (net case 
error). This represents a net $326 million overaward to the 2.53 million Pell Grant 
recipients, dr 13 percent of the .$2.4. billion awarded to students in 1982-83. An 
estimated 41.5 percent of the recipients were given too large an' award. ^ These, 
overawards averaged $444, or k prdgram-wide $465 million in overawards. An 
estimated 21.2 percent of the recipients were given too little money. These 
underawards averaged $259, for a program-wide $139 million in underawards. • 

As Table 2-3 depicts, many of the overawards and underawards were'^bstantial. 
For exarflple, rl4.1 percent of all recipients, an estimated 36Q,'000 students, received 
pver $5,5p jnore than 'they should haye, while /3^8 percent ot all recipients (96,lo00 
recipients) received over $550 less than they should have. * ' . * ; . - 



TdJbJe, 2-4 depicts the, distributxDrv of case errorv across overawards (bothJ 
•eligibles .and ineligibles) and underawards. - Examining the 41^5 percent of recipient §!l 
who received overaward^, 29.8 percent of all recipientsi received more of a Pell Gfant 
Vthah thi^y should, but jstill were eligible for some inward. HoVever, 11.7 percent of all 
recipients (approximately 300V000 students) should have received no award. These 
ineligibles wer& overawarded by an average- of 1^824 'each and contribute more than 
half; of all funds that were overaWarded;.^ Ted- percent bf:alLp 
these ineligibles. ' ' [ ^ : i'--: / . : . * : . " 

2.2 OVERVIEW OF STUDENT ERROR - ' ' . 

The student component of* overall error is that which is attributable to 
discrepancies in Pell Grant application values submitted by students and their parents. 
Absolute error attributable to students and their parents is roughly equal to absolute 



TABLE 2-2 



AMOUNTS AND RATES OF ERR6r WITH SELECTED 
TOLERANCE LEVELS OF PJO ERROR 



- Program-Wide Estimate ' Estimated Cases 

Absolute Net with Error 

($ Millions) ($ Millions) (%) 



Case Error ($) 



+ 2 605 326 '62.7 

+ 25 r, 59& " 320 , 55.8 

+ 50 591;r ^ 318 t^3.7 

+ 100 4 577^ ^ 316 t^lA 



Student Error ($) 



+ 2 . 328 . 217 39. 

+ 25 ^ 323 212 36.0 

+ 50 ♦ 318 209 30.9 

+ 100 307 207 25.8 



Institutional Error ($) 

+2 321 99 33.5 

+ 25 318 98 27.5 

+50 315 99 2k.3 

+ 100 307 100 20.6 



TABLE 2-3 

PERCENTAGE OF CASES WITH ERROR BY DOLLAR RANGE 



Dollar Range 



Case Error 
(%) 



Case Error Treating Cases with 
SEP/FAT/Invalid SAR Errors as EUgibl< 
(%) 



SSI 


cinci 


more 


1 /t 1 


, - 11.2 


7SI 


to 


ssn 


9 /i 
O. 'f 






to 




5.1 


5.2 


lUl 


to 


150 


3.2 


3.3 


^1 


to 


luu 


^.3 


't.5 


'yc 


to 






'f.5 


3 


to 


2j 


1.7 


1.8 


A 


to 


2 


37 . 3 


39.0 


'\ 

\ 


TO - 


Aj 


1.7 . 


1.9 


' 26 


to - 


50 




2.7 


51 


to - 


100 




3.3 


101 


to - 


150 


2.2. 


2.3 


151 


to - 


250 


3.1 


3.2 


251 


to - 


550 




*.8 


551 


and less 


3.8 


3.9 
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TABLE 2-<> 

AND RATES OF OVERAWARD AND UNDERAWARD 



Case Overaward Case 

All Eligibles Ineligibles Underawarc 



J 



^^65 220 li^k - 139 



290 82't - 259 

f 

'H.5 % 29.8 % 11.7 % 21.2 % 

1 19 % 9 % 10 % 6 % 
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error attributable to schools and thus comprises about one- the program-wide 

absolute error reviewed in the previous section. However, the ratio of overaward6 to 
underawards is substantially higher for student error than institutional error. There- 
fore, net error associated with students is roughly twice the net error attributable to 
schools, comprising about two-thirds of program-wide net error. In Chapter ^, a 
detailed examination of student error and its component parts is presented. In this 
overview section, the types of student error are defined and highlights of the findings 
for 1 982-83 are presented. 

2*2*1 Types of Student Error 

Student error is defined as the difference between' what the recipient's award 
should have been using correct student and institutional data and what the award 
would be using correct institutional data but the student data as reported on the 
recipient's application. Effectively, for any student, it is the error attributable to 
discrepancies in application values as measured on the application that was the basis 
for the student's award determination. 

An examination was also made of the marginal contributions of each data 
element on the application to student error. The marginal contribution of a data 
element was calculated for each student by first determining the amount the student 
would have received using the correct institutiorf§i,data and the student data reported 
on the application. Then this determination was repeated except that the "best value" 
of the data element in question was substituted for student-reported value of that data 
element. The marginal error contribution of the data element was then set equal to 
the difference between these two amounts. Thus, each marginal student errdr is a 
measure of the affect on award of a student's reporting that particular data element 
incorrectly. ^ 

2.2.2 Measures of Student Error 

Table 2-1 shows that net student error averaged $86 per recipient for a total 
.program-wide net student error of $217 million (9 percent of Pell Grant dollars 
awarded It in 1982-83). The $217 million was the net overaward resulting from $272 
million in overawards to 30.6 percent of the recipients (770,000 $tudents) and $55 



million in underawards to 8.8 percent of the recipients (220,000 students). Thus, 
nearly one million students (39.^ percent of the Pell Grant recipients in 1982-83) had 
application errors resulting in $2 or more of award error each and totaling $328 
million. 

^ The second panel of table 2-2 displays the effect of changing the $2 tolerance on 
the amount of student error and the {Percentage of cases wit^yerror. As can be seen, if 
error is defined as only those cases where application form discrepancies result in 
award errors in excess of $100, student error was estimated to occur in only 25.8 
percent of the cases (650,000 students). However, when the error tolerance is 
increased from $2 to $100 the program-wide estimates of error decreased by only 5.5 
percent and 2.8 percent for absolute and net errors, respectively. 

The greatest contributor to student error was found to be the incorrect 
classification of a student's dependency status. An estimated 5.1 percent (129,000) of 
all recipients originally awarded as independent students were determined actually to 
be dependent, while A percent (10,000) of all recipients initially awarded as dependent 
students were determined to be independent. The effect of these incorrect classifica- 
tions was an estimated $6^ million in both net and absolute overpayments. 

The second greatest contributor to. student error was the incorrect reporting of 
"Other Nontaxable Income." This data element includes child support, the untaxed 
portion of unemployment compensation, the interest/diviclend exclusion, and other 
welfare (excluding AFDC) benefits^ among other items. The estimated marginal 
effect of discrepancies in this data item was $^6 million in overpayments. Other 
major contributors to student error were discrepancies in reporting household size, 
number in postsecondary education, home equity, dependent student's (and spouse's) 
income and assets, and adjusted gross income of independent students and parents of 
dependent students. 

2.3 OVERVIEW OF INSTITUTIONAL ERROR 

Institutional error is defined as the difference between the award that 
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the school gave the student and what the award should have been, given the student's 
SAI (regardless of whether the student's application data were correct) and the rules 
and regulations governing the Pell Grant program. In Chapter 3, a detailed 
presentation of institutional error is given. In this section, the components of 
institutional error are defined and estimates are given for 1982-83. 

2.3.1 Types of Institutional Error 

Institutional error is decomposed into two parts: eligibility error and disburse- 
ment error. Eligibility error occurs when a recipient is ineligible for an award due to 
noncompliance with one or more of twelve regulatory criteria. Disbursement error 
occurs when a student receives the wrong amount of award due to calculation or 
. accounting mistakes on the part of the institution or the use of an incorrect cost of 
attendance or enrollment status for the student. 

^ Eligibility error can only be an overaward since it is defined as giving an award 
to a student who should not have received one. As a result, the net and absolute 
eligibility errors are equal. Disbursement error includes overawards and underawards, 
so net error is less than absolute error. As will be shown, the absolute dollar value of 
underawards due to disbursement error actually exceeded overawards due to disburse- 
ment error in 1982-83. 

2.3.2 Measures of Institutional Error 

Table 2-1 shows that an estimated 15.7 percent of Pell Grant recipients for 
1982-83 were pverawarded and 17.8 percent were underawarded due to"^ institutional 
error. However, the mean overaward exceeded the mean absolute underaward by more 
than two to one. This is because overawards are usually eligibility errors, which means 
that student's entire grant is in error. As a result, net program-wide institutional 
error was positive, and equal to $99 million or an average $39 overaward per recipient. 
Absolute error, the sum of overawards and underawards, totaled $321 million and 
averaged $127 per recipient. 

Table 2-5 presents a breakdown 6f the components of institutional error. An 
estimated 5.2 percent of all Pell. Grant recipients (an estimated 130,000 students) 
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TABLE 2-5 

dilillPONENTS OF INSTITUTIONAL ERROR 



ABSOLUTE ERROR 



Mean Error 
per Recipient 



Eligibility Error 

Insufficient Program Length 
Nondegree Program 
Not Parent Institution 
Default on Loan 
Less Than Half Time 
Unsatisfactory Academic Progress 
Invalid SAR 

No Statement of Educational Purpose 

No Financial Aid Transcript 

No SAR in File 

Has Bachelor's Degree* • 

Not a Citizen or Eligible Noncitizen** 

Disbursement Error 

Cost of Attendance Error 
Enrollment Status Er ror 
Calculation/Accounting Error 



2 
3 

37 
2 



77 
19 
58 
22 



Mean Error 
per Recipient 
W/Error 
($) 

1,078 
79 
1,031 
1,018 
505 
601 
656 
855 
1,043 
1,168 
1,725 



255 
180 
260 
182 



Program- 
Vide 
Estimate 
($ MilUons) 

.1 
13 

5 
A 

1 

6 

8 
10 
95 

4 



195 
W 

147 
57 



Cases 
V/Error * 
i%) 

5.2 

.03 

.49 

.21 

.03 

.09 

.38 

.36 

.39 

3.2 

.10 



30.2 
10.8 
22.3 
. 12.3 



*Less than $1. 

**There were no instances of these errors in the sample used. 



NET ERROR 



Mean Error 
perR«lple„t 
wl 


Mean Error 
per Recipient 
V/Error 


Program- 
Estimate 

iK Uillinn«1 


JO 






* 


79 


.1 




1 ,051 


13 


2 


1,018 




m 




,4 


* 


601 


1 


f 


656 


6 


3 


855^ 


8 


4 


1,043 


10 


37 


1,168 


9^ 


2 



1^725 


4 

— 


-15 


-50 


-38 


-8 


-76 


-21 


-16 


-71 


-39 


9 


77 


24 
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should have been ineligible for any award due to noncompliance with one or more of 
the twelve eligibility criteria listed. Clearly the largest group, "No Financial Aid 
Transcript (FAT)," dominates the causes of ineligibility, exceeding the next most 
frequent cause by almost seven to one. Other than FAT error, very little' discrepancy 
based upon general eligibility requirements was found. 

The value of disbursement error underaWards exceeded overawards and an 
estimated net $38 million in disbursement underaward error occurred in 1982-83. This 
occurred because institutions generally understated cost of attendance or enrollment 
status. More than one-fifth (22.3 percent) of all recipients received awards based, in 
part, on the wrong enrolfment status, while 1 in 10 received awards based on the wrong 
cost of attendance. 

2A EFFECT OF FAT/SEP/INVALID SAR ERRORS 

^As shown in Table 2-5, the lack of a Financial Aid Transcript (FA J) on file for a 
transfer student was the largest factor of the eligibility error component erf 
institutional error. Error associated with the absence of a FAT, lack of a signed " 
Statement of Educational Purpose (SEP), or use of an invalid Student Aid Report 
(SAR), such as a photocopy, is different from the error associated with the other nine 
criteria. The first three are failures to follow procedure and are indicators of 
potential error. The last nine are matters of fact that make the student ineligible. In 
this section, an overview of the effects on error of ignoring FAT, SEP, and Invalid SAR 
error is presented. 

Table 2^6 is analogous to Table 2-1 except that errors relating to a missing FAT, 
a missing SEP, and not having a yalid^SAR on file were ignored. Progr^^m-wide net 
case error dropped from the estimated $326 million overpayment reported earlier to 
$226 million overpayment. This drop of $100 million may be considered the marginal 
impact of FAT/SEP/Invalid SAR error. In all, net institutional error dcopped $112 
million from an estimated $99 million in 6veraward to $13 million in underaward.' 
However, net student error increased by $1^ million. This was because students with 
application error, but who had institution-ba^ed eligibility error were, by the definition 
of student error, not included in the student error total. When FAT/SEP/Invalid SAR 
error was ignored, students who had both FAT/SEP/Invalid SAR error and application 
data error contributed to the student error calculations. 



TABLE 2^ - * 

A SUUiyiARY OP AUOUNTS AND RATES OF ERROR 
TREATING CASES WITH SEP/PAT/INVALID SAR ERRORS AS ELIGIBLE 



ABSOLUTE ERROR 



NET ERROR 





Proftram-Wide Estimate 


Mean 

Error per 


Cases 


Mean 
Error per 
Recipient 




ProKram-Wide Estimate 


Mean 
Error per 


Cases 
W/Errof*> 
(%) 


Mean 
Error per 
Recipient 


Error 


($ UiUiofu) 


{% of 
Awarded)^ 


Recipient 
($) 


W/Error*> 
{%) 


W/Error 
($) 


Error 


($ Millions) 


Awarded 


Recipient 
($) 


W/Error 
($) 


Institutional 


217 


9 


86 


30.6 


281 


Institutional 


-13 


.5 


V5 


30.6 


-17 


Student 




U 


137 


41.1 


332 


Student 


231 * 


10 


91 


41.1 


222 


Total 


562 


23 


223 


61.1 


364 


Total 


217 


9 


84 


61.1 


141 


Case 


517 


21 


204 


61.1 


334 


Case 


226 

• 


9 


89 


61.1 . 


146 






OVERAWARD ERROR 






•> 


^ UNDERAWARD ERROR^ 








ProRfam-Wlde Estimate 


Mean 
Error per 


Cases 
W/Error*> 
{%) 


Mean 
Error per' 
Recipient 




\ ' 
Pro(vam-Wide Estimate 


Meui 
Error per 
Red^ent 


W/Errar>> 
(%) 


Mean 
Error per 

R«:iDiMt 

W/Error 

($) 


Uror 


($ UiUions) 


Awarded)* 


Recipient 
B) 


W/Error 
($) 


. Error / 


($ Millions) 


Ami^kl)> 


Institutional 


102 


4 


W 


12^2 


331 


Institutional 


-115 




-46 


ft 


: -248 


Student 


288 


12 


• 114 


32.0 


355 


Student 


-57 


2 


-23 ^ 


9.1 


-250 


Toul 


390 


16 


154 


39.1 


394 


Total 


-173 


7 


-68 


^2.0 


r3?0 


Case 


371 


15 


147 


39.1 


375 , 


Case 


-145 


6 


-57 


22.0 


-261 



to 
I 



•Amount of Pell awards is $2.4 bilUon for 1982-83. 
Error is defined as a discrepancy of plus or minus $2 from the best award. 
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Overall, an estimated 61 percent of all recipients had award error in excess of $2 
when FAT/SEP/Invalid SAR error was ignored. An estimated ^2.3 percent of all 
recipients had award error in excess of $100. Further, as shown" in Table 2-3, 11.2 
percent (260,000 students) had overawards in excess of $550. 

Ulnless explicitly stated otherwise, a ^missirig FAT or SEP, or an invalid ;SAR is 
considered to be an error and . the recipient is deemed ineligible thr-otighout the 
remaindifir of this report. Thus, most subsequent findings are based on the assumption 
that procedural errors make a recipient ineligible, as required by 'the current 
regulations. . / 

s ■■■'■■^'^ 

2.5 CONCLUSIONS . { 

The findings which are overviewecf 'In'-this chapter have the following 
irjiplications. , • 

Since ^one-in-five program dollars were misaliocated, the need for 
corrective actions remains. . . ,4^. 

Categorical .^eligibility errors occurred with ^a low incidence and indicate , 
less heed *for''correctiVe actions. ■ - 

Institutipnal disbursement errors were generally undera wards; however, the 
magnitudes of these require prompt attention^ ^ 

.Student error, which, was predominantly overawarding erroT", accounted for 
more thiin ($0 percent of t^^ ^ 

Problems with incorrect dependency status were the major source of 
student error. • - >rv 



\ -CHAPTER ^ 

. . ■•. ' . \ ■ U'^^ .' ^ ■■ .- 

This chapter presentsNpstimates ot institutional error ih the Pell ■ Grarit^program 
for 1982-83. .Institutional error is ^e^^^ in tern^s of the componeri^ and 
possible causes. Sig^icant findings presented, in this chapter'include:- •^ - • 



With the exception pf a lack of a Financial Aid Transcript <MT) being on 
file, institution-related eligibility error was very rare (under 2 percent of 
all cases). . " . , " 

Disbursement error occurred in over 30 percent of all cases. Disbursement 
errof was composed of cost of attendance error (11 pfercent error rate), 
enFiSllment status ef ror (22 percent error rate), and calculation/accountine 
'efroj (12 percent error rate)|j I J , 

Eligibility error (primarily lick, of an FAT) rei^ulted in overawards totaling 
an estimated $l't2 million. Disbursement iertor, however, resulted in a net 
underaward' of $38 million. 



Excluding SEP/FAT/Invalid SAR error resulted in net preigram-wide 
institu^onal error of $-13 mUli<^rt. That is, institutional error resulted in 
students receiving $13 million leite than they were entitled to receive. 

Error. r^tes differed significantly' across insti-fotipnal control and type with 
proprietgy;^^ having generally higher progensities to make an error. 

Clock-h^S;; term-type, and t, credit me^^^f^t schools, which are 
overwhelri^hgly. proprietary, also fexhibitejj^her^rror r^tes. 



• . . Institution^haracteris'tics other tljiifi type, control, term type, and credit 
mea^urffljlfivtype were not good explanations of causes of error. 

3.1 NAt^^VND TYPES OF INSTITUTIONAL ERROR 

t'^ 9^^P^ 2 institutional error was defined as the, difference between' the Pell 

^""^^^ and the amount the school should have ' given the 
student with the Studlgt^'Aid-Index (SAI) the school Jiad in its files.' In this section we 
refine this definition If^jlpclude the component [iarts of institutional error. 



..^ two major, types oi ^ih|titufionai^ 3-1: 

V EUgibility error occurs' when a recipieii'^s ineligible- for an :awaV4 '^ noiicompli- 
. 'ance vwi^^ one or' more of the twelve- criteria shown ih Figure 3-1. We have 
operationally defined eligibility error as the amount of .'jKe va^ ,by the , 

student, asSujTiing that the institution calculated thie : av/ard': c^ using, the 

information available? to it. As is t|ie case for other eligibility errors, eligibility error 
du|e to a missing F^T is defined as the entire amount disbursed. Since schools, are 
allowed to make a first disbursement without a transfer stu<ieht*s FAT^^ definition 
. of error includes some portion— that after the first disbursiemeM--,for^^)^^ there is no* 
institutional liability. v: . ' . 



The ej^biiity errors, along with their definitions, are listed below. 

• * K^ufficent Program Length— The student's academic program miist be at 

least six months in duration. : 

.... ' • Nondegree Program— The student's academic program , must lead to a^ 
' certificate or associate, bachelor's, or undergraduate professional degree. 

' • Not Parent Institution— the Pell Grant disbursement for a student 

' . -^^^^ attending consortium programs must be made by the student's parertt 

: * . institution. , . ^ ; ^' : ^ 

V . . • Default on Loan— No student* may receive a Pell Grttnt'if in default on a 
, Guaranteed Student tocin or a National Direct Student Loan awarded by 
that .school lar ia^^d^ on a repayment agreement for any Title IV 
- ; progratti^frdm that schooL ' . 

• , ^* Less Than. Haitf^^ student must be enrolled ai: least one-half time 

in an eligible ppogram. . . *^ » 

. \ /. : r \ • Unsatisfactory Academic Progres^-The ' stud^t^musj' be meeting the 
" school's own criteria for satisfactory progress. *i . 

^ /\ • Invalid Student Aid- Report (SAR)— The school must keep an original of the 
' V , most current Student Aid Report on file. ^ 

ii, ' No Statement of Educational Purpose (SEP)— The student must sign a SEP 
. v indicating the intent to use the Pell Grant for education-related purposes 
: and the school must keep the copy on file. 

No Financial Aid Transcript (FAT)— The school rfiay not make a second V 
■ • . ;{disburs^m^ Pell Grant funds to a transfer student without ieceiving a^* 

: - \FAt fro^^^^^^ 

• . - No SAR in File— -The school must have a SAR for ail Pell Grant recipients. ^ 

\: : ' '• ' 3-2 ■ ■• ' , 

erIcm s di 



INSTITUTIONAL ERROR: 



Eligibility 
Error 



Disbursement 
Error 



Insufficient Program Length 
Nondegree Program 
Not Parent Institution 



Cost of Attendance Error 

• ' Enrollment Status Error 

• Calculation/ Accounting Error 



• Default on Loan or Payment 

• Less Than- Half -Time — .^^^ - . ^ 

• Unsatisfactory Academic Progress 

• Invalid SAR 

• No Statement of Educational Purpose 

• No Financial Aid Transcript 

• NoSARin-File 

• , Possession of Bachelor's Degree 

• Not a U.S. Citizen or Eligible Non-titizen 




V 
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FIGURE 3-1 



THE COMPONENTS OF INSTITUTIONAL ERROR 
IN THE PELL GRANT PROGRAM, 



• Possession of Bachelor's Oegrefe-The student may not already have a 
bachelorls (or equivalent) degree. . . 

• Not a U.S. Citizen-Students must'be U^S. citizens or eligible non-citizens. 

Disbursem^t error occurs when a student receives the wrong amount of award' 
part of-,$he institution in determining cost of attendance ior 
enr'ollmeht s;tatus, calculating the award amount, or accounting for disbursement 
activity as defined by the Pell Grant program. Thus, disbursement 'error can be 
reported as a whole-the difference between the amount disbursed and the amount of- 
the award calculated using the best values for cost of attendance and enrollment 

^"^ '*'^_^"^^J*"°^"_1°J='^^^ vaifaMes, assuming the 

studeM is eligible~or broken down into its three component pSts. These component 
parjs are listed in Figure 3-1. The sum of the errors for the parts is aot the same as 
the whole becaus^1;fe.£ormulas used for each part isolate its error and do not properly 
capture interactions with the other parts. 

We have used best values for the complete academic year in determining cost of 
attendance and enrollment status errors. There is, however, nothing in the current 
_ regulations requiring continued monitoring of cost of attendance and enrollment status 
by' the institutions as long as their initial figures were reasonable at the time the 
expected disbursement was determined. We are using the approach indicated because 
it is the only way to arrive at uniform and consistent estimates of error, even though . 
some part of each. -errpf may not result in institutional liability. 

3.2 INSTITUTIONAL ERROR HNDINGS 

.. /- Table 3-1 summarizes institutional error for 1982-83. Overall, an estimated 33.5 
-percent ^5,000) of all Pell Grant recipients had incorrect awards due to institutional 
error. The likelihood of-an underaward error (17.8 percent) slightly exceeded that for 
an overaward error (15.7 percent), but the average overaward was more than twice the 
size of the average underaward. As a result, overawarded dollars exceeded 
underawarded dollars by an estimated $99 mUlion or $39 per Pell Grant recipient. This 
net overaward equaled approximately percent of program expenditures- for 1982-83. 
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TABLE 3-1 



AMOUNTS AND RATES OF INSTITUTIONAL ERROR 

I 



Absolute Net 

Institutional fristitutlonal Institutional Institutional 

E""or ' Error Overaward Underaward 

Program- Wide Estimate ($ Millions) 321 99; 2U " -111 



Mean Error per Recipient ($) 127 39 g3 
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Mean Error per Recipient with 
Error ($) 379 nZ 528 -21^7 

* . ' ■ 

Percentage of Cases with Error # 
Greater Than $2 33.5 % 33.5 % 15.7% 17.8 % 

Error as a Percentage of OoUars 
Awarded 13 % % 9 % 5 % 



The range of institutional error is shown in Table 3-2. From this table it can be 
seen that more than 22 percent of the recipients had institutional error (o vera wards or 
underawards) in excess of $100. Also, 6.8 percent (170,000 students) had overawards in 
excess of $550 and 3.5 percent (90,000 students) were underawarded in excess pf $550. 

The second column of Table 3-2 lists the range of error when the lack of a S^P, 
FAT, or a valid SAR on file at the school is not considered ^an error. Drppping these 
eligibility criteria decreases the proportion of recipients with institutional error to 
30.6 percent, or 775,000 students, and significantly decreases the number of the cases 
with large overawards. Table 3-3 replicates Table 3-1 excluding SEP/FAT/Invalid SAR 
error. The effect of dropping the large overawards associated with these three 
eligibility errors is to drive net institutional error to an insignificant level, a negative 
$13 million. However, institutional error still exists, as shown in the $217 million 
absolute sum of overawards and underawards. 

3.2.1 Eligibility Errors ^ \ 

Eligibility error can only result, in an overaward since it is defined as giving an 
award to a student who should not have received one. Table 3-* displays the levels, 
frequencies, and payment consequences of each type of eligibility error. The sum of 
the components of eligibility error is very slightly greater than the overall eligibility 
error because almost 2 percent of the cases have more than one eligibility error. 

< 

^ It is clear that with the exception of no FAT on file, institution-related 
eligibility error is nearly inconsequential in the Pell Grant program. In total, 
eligibility criteria other than the FAT were not satisfied in fewer than 50,000 cases 
(2 percent) and resulting dollar error was under $50 million. The last two criteria- 
bachelor's degree and citizenship— were always satisfied for the sample of recipients. 
Given the low frequency of each component it is a good presumption that schools are 
generally doing a good job in ensuring compliance with the regulations regarding 
eligibility. The lack of a Financial Aid Transcript is somewhat rnore of a problem ancf 
is explored further elsewhere in this findings report and in Volume 2, Corrective 
Actions. 
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TABLE 3-2 

PERCENTAGE OF CASES WITH INSTITUTIONAL 
ERROR BY DOLLAR RANGE 



Dollar Range 




Institutional Error 

. (%) 


551 


and more ^ 


6.S 


251 


to 


550 


2.3 


151 


to 


250 


\A 


101 


to 


150 


1.0 


51 


to 


100 


1.8 


26 


to 


50 


1.^ • ' 


3 


d 

to 


25 


1 .2 


2 


to 


2 


66.5 


3 


to - 


25 


2.8 


26' 


to - 






- 51 


to - 


: 100 




- 101 


to - 


150 


1.8 


: 


to - 


250 


2.2 


- 251 


to - 


550 


3.3 


- 551 


and less 


'3.5 



46 



Institutional Error Treating 
^SEP/FAT/InvaUd SAR Cases as EUgibk 
(%) 



3.3 
2.1 
1.3, 
0.9 
1.8 
1.5 
1.2 
69.4 ' 
v2-9; 

2.0 
1.9 
2.3 

3.6 
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TABLE 3-3 

AMOUNTS AND RATES OF INSTITUTIONAL ERROR 
TREATING CASES WITH SEP/FAT/INVALID SAR ERRORS AS ELIGIBLE 



0 



Absolute Net 
Institutional . Institutional Institutional Institutional 
Error Error Overaward Underaward 



Program- Wide Estimate ($ Millions) 217 



-13 



102 



■115 



Mean Error per Recipient ($) 



86 



-*6 



Mean Error per Recipient with 
Error ($) ^ 



281 



-17 



331 



-2*8 



Percentage of Cases with Error 
Greater Than $2 



30.6 % 



30^6 % 



12.2 % 



ISA % 



Error as a Percentage of Dollars 
A Weirded 



9 % 



.5 % 



5 % 
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TABLE 3-4 

THE COMPONENTS OF ELIGIBILITY ERROR 



Cases With Error 



Insufficient Program Length . 

Nondegree I^rogt^m ■ 

Not Parent Institution 

Default on Loan 

Less than Half Time 

Unsatisfactory Academic 
Progress ' 

Invalid SAR h 

No SEP 

No FAT , ,': . 

No SAR in File <^.t "* 
Not a Citizen \ 'M fe 
Possess Bachelor's Dir'^ 



■ A 



:.:-\ \ 
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Overall Eligibility Errot j;. ;' ■ v ^li^^' ■ - . 130 , 000 



1,078 



%Of AH 
Recipioits 


Number 


Mean Error 

per 
Recipient 

($) 


Program-Wide 
Error Estimate 
($ Millions) 


0.03 


1,000 




79 


n 1 




12,^00 


1 


,051 


1 


o.2i ;'v 


5,200 


, 1 


,018 


-/ 


0.03*?:X:,;:- 


, ^ .1,000 




505 




0.09 


1,600 




601 


1 


^ 0.38 


9,100 




656 


6 












^r.0.36 


9,300 




855 


8 


^ :Q-39 


9,800 


1 


,0^3 


10 


%^'3.'20 


81,000 


1 


,168 




'■jta/10 


2,^00 


1 


,725 






0 




0 


d 


^';-o.a; 


0 




0 


0 



l't2 



3.2.2 Disbursement Error 

Disbursement error was much more common than eligibility error, occurring in 
30 percent of all cases. Table 3-5 is a display of the level, frequency, and payment 
consequences of the components of disbursement error. Both the frequency and 
overall value of underawards exceeded overawards, so that in balknce disbursement 
error resulted in those recipients with disbursement error receiving $50 too little, on 
average^ « . ' 

■ ' i ■ ' 

Enrollment status error was the most frequent disbursement error and generally 
resulted in underawards. As shown in the bottom panel of Table 3-6, enrollment status 
error was more than twice as likely to result in an underawarcj than an overaward. 
Thus, schools tended to understate the enrollment status of recipients (i.e., to, award 
on the basis of the student being enrolled for fewer credits than was actually the 
case). ^ 

While the net cost of attendance error was negative (underaward), approximately 
the same number of recipients received erroneous overawards as received erroneous 
underawards. In 1 percent of all cases (25,000 students) a school's understating a 
recipient's cost of attendance resulted in that student's receiving at least $550 less 
than he or she should have. 

Calculation/accounting error generally led to students' receiving more than they 
W^l!'i0*h/^°^^ ^" ^^^^^ ^^^"^ ^l^is error, the average student received $77 more than he 

VsJy^-^-M^°^ she should have. In all, an estimated $2* million tqo much was awarded because of 
calculation/accounting error. 



Overa][l, disbursement error was more likely to result in an underaward as 
compared to an overaward and generated a net $38 million decrease from what should 
have been awarded to students in 1982-83. 

3.3 EVALUATION OF POSSIBLE CAUSES OF ERROR 

We presented program-wide estimates for institutional error and its major 
components in the previous section. Examining differences in the level and frequency 
of these errors across institutional types, procedures, and practices can provide 
information about the possible causes of these errors. / 
Q 3-10 
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TABLE 3-5 

THE COMPONENTS OF DISBURSEMENT ERROR 



Cases With Error 



Net 

% of all Mean Error 

Recipients Number ($) 



Disbursement Error 30 , 2 

■ .. ^ 
Cost of Attendance 

Error 10.8 

Enrollment Status 

Error 22.3 

Calculation/ 

Accounting Error 12.3 



761^,000 
273,000 
561^,000 
311,000 



-50 
-76 
-71 

77 



P rogram- Wide Error Estimate s 
Absolute 
Error Net Error 

( $ Millions ) ($ Millions) 



195 



f9 



1^^7 



57 



-38 



-21 



-39 



21^ 




TABLE 3-6 

PERCENTAGE OF ^ASES WITH DISBURSEMENT 
ERROR BY DOLLAR RANGE 



Dollar Range . 



Total 
Disbursement 
> Error 
(%) 



Costbf 
Attendance 
Error 
(%) 



Enrollment 
Status 
Error 

(%). 



Calculation/ 
Accounting 
Error 
(%) 



I. 



551 


and 


more 


2.7 


.3 


251 


to 


550 


la? 


.5 


151 


to 


250 


1.2 




101 


to 


150 


1.0 • 


,■ .3 


51 


to 


100 


1.9 


1.1 


26 


to 


50 


' lA 


1.7 


3 


to 


' .25 


1.3 


.5 


-2 


to 


2 


69.8 


89.2 


-3 


to 


-25 


2.9 


.7 


-26 


to 


-50 


2.5 


l.'f 


-51 


to 


-100 


2.2 


.8 


-101 


to 


-'150 


1.9 


.6 


-151 


to 


-250 


2A 


.3 


-251 


to 


-550 


3.3 


1.1 


-551 


and 


less 




1.0 



Summary. 

Overawacds 

No Error (± $2) 

Undera wards 



UA 
69.8 
18.7 



89.2 
6.0 



1.7 


1.0 


1.7 


1.2 


;9 


1.1 


.9 


1.1 


1.0 


1.2 


. A 


.9 


.5 


.9 


77.7 


87.7 


1.5' 


1.1 


1.8 


.7 


1.8 


1.1 


2.0 : 


.5 


2.1 


.5 


3.5 


.7 


2.5 


.3 


7.1 . 


7 A 



77.7 
15.2 



87.7 
1^.9 
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:^iya\ysis methods w§re utilized to perform a causal analysis: . simple 
bivariate tabl^sr^land multivariate regressions. Before we discuss the results (in 
Sections 3;|.2 , apd 3.3.3) we present an enumeration of possible 'causes in the next - 
. section (3.3.1 ). •:'; ■*', ' , .v;;., \ 

3.3il Enumeration of Possible Causes 

Since the purpose of the causal analysis is to support the development of 
manlgement corrective actions, ij is^, necessary to characterise^' the possible faI:tors . V 
which explain differences ir^e^rrpr Fates as discretionary or'ihfrinsic. Intrinsic, factors ' 
are characteristics which cahS not be influenced or altered, as part of management 
action. Intrinsic factors would, include organizational cpnstraints, characteristics of 
the population served, and environmental, or background considerations;;' ti'fscretionary 
factors would include variables which describe the system, process features used in 
administering the Pell Grant program, and indicators of management practices. Table 
3-7 is a listing of the institutional characteristics which we reviewed as potential 
causes in the following two sections. This list is restricted to variables which meet 
two conditions: ■ " 



The variable must exhibit sufficient variation across the institutions in the 
sample. 

The variable must be expected to be a cause of one or more of the six 
components or measures of institutional error. 



This first condition is jan objective criterion. The second criterion is more subjective 
since we are hypothesizing whether or not a relationship is likely to exist. 

3.3.2 Simple Bivariate I Causal Analysis 

In this section we examine the error ratesV (cases with error, .cases with 
overaward errors, and cases with underaward error) for the following six error 
measures: / 
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• Overall institutional error 

• Eligibility error 

• Disbui-sement error 

• Cost of attendance error 
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TABLE 3-7 V . . 

INSTITUTIONAL CHAIIACTERISTICS USED TO ANALYZE 
POSSIBLE CORRELATES OF ERROR 



Characteristics 
Intrinsic 

Type of Institution 

Control of Institution 

Term Type . «^ . 

Credit Measurement Systeim / 
. ■ ./^^ of Recipients as Ratio 

; ■ of Enrollment 

; ' Administrative System 



Discretionary 

^Method of Disbursement 
Fr,equency of Disbursement 
Method of Award Calculation 
Performance of Own Validation 
Frequency of Award Recafcufation 
Routin€«Reverification of fi^ari^ 

Calculation ^- ■ '^aV- 

Personnel Used for Award 
ct'v Calculation 




0 
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• ^rir^llment status error " • " ' 

• - Calculation/aceountingierror. - „ • " . 

,• . * " • " • ■ ■ » ' "■ ■ 

The institutional^chfiiracteristics enumerated in Table 3-7 were tested to. 
determine whether they could explain variation in error rates across institutions and 
thus., be considered as possible c^ses of these errors. Not all institutionar 
characteristics )wer^ examined^ ^s poss causes for all error measures/ Rather we 
selected tho^e combinations we thought were' most Ifkely to show rc^lationships. Our 
decisions were' based on our . experience with the early Pell studies, knowledge of the 
delivery system, and an understanding of the Pell Grant award determination process. 
The * possible causes . of institutional ^errpr due to intrirtsic and ""discretionkry 
; institutional characteristics are discussed separately, i^-V ' 

to Ghiaract^istics . 

Table 3-8 displays rates of institutional: Vrror across the six categories of ^ 
• ipstitutiorial type and control; The differences in .the proportion of cases with error 
are statistically significant. Overall 33.5 percent of the cas.es had institutional error, 
about 16 percent with overawards and 18 {Sercent with undep^wards. Proprietary 
schools had error^ rates of ;over^^^ 2-year schools ^ showing more 

oVerawards and less than 2-year schools showing more underawards. The two ^ 
categories of ^-year schools had error rates pf'about^28 and 22 percent^, making them- 
the least error-prone of the six categories. While public Vyear schopls had nearly 
equal overawards and underawards, such was not the case for private, ^-year school^ 
where there were 1.6 overawards for each underaward.^ Of particular importance 
would be the high error rate (^6 percent) for public 2-yeair schools because the/ 
account for such a significant proportion (2^ percent) of the Pi^eipient universe. 

v . Table 3-9 examines the fre;;piency of eligibility error across the six type and 
control . categories. The overall rate 'dfi-error^ was 5^^^ percent with four ofv the 
categories fairly close to this overairVate. THe two exceptions were the two, 
proprietary school categories where eUgy^imy errbf Tates exceed^^ ^ • . 

' ' - The pattern which emerges from^ these two tables, , i.e., "p^ 

more error, is also true for disburienient error (Table 3-10), enrollment stat^s'^r^^^ 

^^^'^^^.^"M;^.^nd calculatton and accounting error (Table 3-12)^ With the exception of 



> TABLE 3-8 

-xPERCENTAGE OF CASES WITH INSTITUfioNAL ERROR,BY 

TYPE AND CONTROL OF INSTITUTION ' 

♦ 



Type and Control 


' Cases' .• 
W/Error* 
(%) 


XBases. 
W/Overtward . 


Public^ Year 


• 28.3 




Public 2 Year ■ 




fit 
• *^ 

17.1 


Private 4 Year 


21.8 


13.6 * 


Private 1 YMr 


33.5.;., 


•J;,: ■ - , . .-V .. 


Proprietary 2 Year /^'^v.. 


51.5 


28.5 


Proprietary Less Than 2 Ye^c, 


56.'7.-':. 


23.2 


All Institutions 


• 33.5 ' 


15*. 7 


*Chi'^<5uare = 17it.201, df f ;5,:£ 


> = .0001, for etvovino error. 











Cases 
W/Underaward 
(%) 

1> 13.5 

p ■■ ; . - 
) ' .. 

28.5 

'-r 8.3 

26.3 
23.0 
33.5 
17.8 
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* TABL^3-9 

PERCENTAGE OF CASES WITH ELIGIBILITY ERROR BY 
TYPE AND CONTROL OF INSTI.fUTIO^" 



Type and Control 


Cases 

. (%) 


Public ^ Ye^r 


5.1 


■'r.ir' ' . J ■ 




Public 2 Year ! 


2.6 


Private ^ Year 


■ 5.9 


Private 2 Year 


3.3 


Proprietary 2 Year 


11.5 


Proprietary Less Than 2, Year 


I't.'t • 


All Institutions 


5.2 



9 



*Chi square = it9.541, df = 5,.2 = .0001, for error/ho error. ' 



^ , . • TABLE 3-10 

PERCENTAGE OF CASES WITH DISBURSEMENT ERROR BY 
, TYPE AND CONTROL OF INSTITUTION, 



> 



Type and Control 



' P&lic H Year" 
Pi^lic 2 Year 

Private Year 

< ■ « ' «. ■ 

■p/ivate 2 Year . 
Proprtetary 2 Year 
Proprietary Less Than 2 Year 
All Instifufions 



Cases 
W/Error* 
' (%) 

16.9 
30.2 

. k7.7 
52.5 
30.2 



Cases 
W/Overaward 
(%) 

10. II- 

15.2 

8.3 ' 

3.9 

11.5 



' Cases 
W/Unideraward 
(%) 

29.5 
8.6 
26.3 
26.2 
W.9 



18.7 



*Chi square = 276.092, df = 5, £ = .0001, for error/no errot.-^ 
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TABLE 3-11 

PERCENTAGE OF CASES WITH ENROLLMENT STATUS 
ERROR BY TYPE AND CONTROL OF INSTITUTION 



Type and Control 


Cases 
W/Error* 
.(%) 


Cases 
W/Overaward 
(%) 


Cases 
W/Underaward 
(%) 


Public,'t Year 


18. if 




13.5 


PubUc 2 Year ' ? 


30. if 


S.S 


21.7 


Private Year 


11^.7 


7.3 


7.5 


Private 2 Year 


11. 


3.8 




Proprietary 2 Year 


k6A 


23.6 


. 22.5 


Proprietary Less Than 2 Year 




10.7 


29.7 


Ail Institutions 


22.3 


- 7.1 


15.2 



*Ghi square = l'f5.537, df = 5, 2 = .0001, for error/no error. 
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TABLE 3-12 



PERCENTAGE OF CASES WITH CALCULATION/ ACCOUNTING 
ERROR BY TYPE AND CONTROL OF INSTITUTION 



Type and Control 


Cases 
w/ error ^ 

(%) 


Cases 
W/Overaward 
(%) 


Cases 
W/Underaward 
(%) 


Public ^ Year 


11.2 ' 


8.0 ■ 


3.2 


Public 2 Year 


11^.2 


9.1 


5.1 


Private * Year 


6.9 


5.0 


1.9 


Private 2 Year 


20.9 


6.1 


l^f.S 


Proprietary 2 Year 


22.5 


3.7 


IS.S 


Proprietary Less Than 2 Year 


23.7 


5.9 


17.7 


All Institutions 


12.3 


7A 





*Chi square = 59.56, df = 5, £ = .0001, for error/no error. 



r 
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calculation and accounting error, public 2-yecir schools were the next most error-prone 
schools. 

A different pattern emerges for cost of attendance error (Table 3-13). Because 
of their high costs of attendance (usually above the Pell maximum of $3,600), errors in 
cost of attendance rarely had payment consequences for proprietary schools or privat;e 
*-year schools. The likelihood of cost of attendance error Was "highest for public 2- 
year schools, followed by private 2-year and public 4-year schools. 

A second intrinsic institutional characteristic, term type, is examined in Table 3- 
14. The differences in rates of error by term type are statistically significant for four 
of the five error measures, cost of attendance being the exception. Clock hour schools 
always had the highest proportion of cases with error for all five error types. 
Semester schools generally had error rates lower than the averages for all institutions; 
however, the differences are small. 

Table 3-15 looks at the influence of the credit measurement system, another 
intrinsic variable, on institutional error. This variable is highly related to term type 
since credit hour measurement generally aligns with semester, trimester, and quarter 
term types while clock hour describes both the term type and the credit measurement 
system. Given the logical relationship between these two intrinsic variables and the 
findings for term type, it is not surprising to see that clock-hour schools had cases 
with error twice as often as credit-hour schools. What is most surprising is that the 
table shows that clock-hour schools were three times as likely to make an 
underawarding error as credit-hour schools. 

The three intrinsic characteristics reviewed so far— type and control of 
institution, term type, and credit measurement system— have something in common. 
The common characteristic, that proprietary schools are most likely to be clock-hour 
schools, makes it hard to separate the independent influences of term typ^, credit 
measurement, and type and control. The results indicated that proprietary/clock-hour 
schools consistently had the highest levels of institutional error. Our data, however, 
cannot suggest why this may be the case. 
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TABLE 3-13 



PERCENTAGE OF CASES WITH COST OF ATTENDANCE ERROR 
BY TYPE AND CONTROL OF INSTITUTION 



Cases Cases Cases 

Tv^.^nwr- 1 ^^^?^* W/Overaward ^ W/Underaward 
. Type and Control (%) (%) (%) 

PubUc if Year ; . ^ 10.2 5.3 i^.s 

Public 2 Yeiar : ; ' - c'.^ .2QA 8.7 11.7 

Private if Year- ^ . 1.7 0.2 -3 1.5 

Private 2 Year - : V ^-l -:^^^^^^^ ^ ^, :0.-0 ' ' 12.3 

Proprietary 2 Year 7 ^ 3T7 ; 2.0 1.8 

Proprietary Less Than 2 Year 6.2 i.Zf if.8 

All Institutions 10.8 't.8 6.0 



^Chi square = 151.867, df = 5, £ = .0001, for error/no error. 
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PERCENTAGE OF GASES WITH ERROR BY 
TYPE OF INStlTtltlONAL ERROR AND TERM TYPE 



Type of Institutional Error 



Term Type 


Eligibility* 


' ' ' ' ' . / . ■ ■ 

Disbursement^ 


^ CU)st of 
Attendance^ 


cnrollmoit 
Status* 


Calculation^ 
Accounting" 


Semester 








18.6 


11.3 


Clock Hour 


. :;'r3l'0;;\-'V.;; 




■ 12A ' 


38.6 


33.2 


Trimester/Quarter. 








29.6 


11.3 


Other 




'35. 7 . 


5.8 


30.3 


' 18.0 


All Institutions' 




■ ■ 30.0 


• 10.8 




1213 


IChi square' ? ?5.67^ df = 


; 3^g = ;0001j fprvprror/n^ error. ■ • ' Jfc^ 






2cKi sqyai4 = 105.69, df 

* •' . . ♦ 


= 3, 2 ,= ,00piv for error/no errors . ? ^ ■ 







2chi ^uare.= 5.3f , df = B^g = .J#^ error. 
■'fChi square = 71.38, df ±.3, £ W^^^^^ for; error /no error. 
^Chi. square = 67.76, df = 3i' tf = 10001, for error/no error. 
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TABLE 3-15 



PERCENTAGE OF CASES WITH INSTITUTIONAL ERROR 
BY CREDIT MEASUREMENT SYSTEM 



Credit Measurement System 
Credit Hour 
Clock Hour 



Cases 
W/Error* 
(%) 

31.0 

61.0 



Cases 
W/Overaward 
(%) 

15.9 

13.6 



■ / ;Cases" : 
W/Urideraward 
" (%) 

15. 1 



^Chi square =111.301, df'= 1, 2 = -0001, for error /no error. 
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An intrinsic characteristic reflecting whether or not the school was an 
independent or single-campus institji^^on or part of a branch-campus system is 
examined as a potential explanatioh of differing error rates iji Table 3-16, The 
differences across administrative structure are not sighificant for the ^eligibility errop 
rate. The error rates rfTctP the overall institutional and disbursment (^rrprs /6/ere highest 
for independent cam'puses -and lower for eithern'ype of branch campus structure. This 
might:;Sbe reflective of differences in the size of institutions and any resuljing 
economies of scale. . 

') < ' ■ 

The final intrinsic characteristic, the ratio' V| Peir to undergraduate 

enrollment, and its relationship to overall rates of Institutional error is shown in Table 
3-17. Institutions with fewer ^h^n 10 percent of :the^^^^ receiving Pell grants 

iTad^the highest overall error ra:te,/f5 percent, with; iincfer awards being 3 tirnfes'^S' likely 
as>o vera wards. The proportion of cases with error (column one) was fairly consistent 
across the other categories of recipient/enrpi)ment ratios, about 30 tp 35 percent. 

Discretionary Characteristics ^ : \ i 

. We now turn to the discretioriary^ ctiarcic^ between these 

characteristics and the various error rnejaSures may be suggestive of management 
corrective actions. i 

While validation is generally thought to focus on student error, we felt that it 
might have some association with levels of institutional error. However, as Table 3-18 
reveals, institutional Validation has little meaningful effect ^ (although there is a 
statistically sigmficant effect for disbursement and cost of attendance errors) on any 
of the six measures of institutional error, with no difference in excess of 5 percent. 
Of course, the 1982-83 program year . had such high levels of Department-mandated 
validation that there were relatively few cases that could, have been subjected to 
institutional validation. .» 

, Another characteristic thought to be potentially a cause of institutional error is 
the skill level of the personnel who calculate the amount of the award. Table 3-19 
displays the findings for this hypothesis for both disbursement error and 
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, TABLE 3-16 

PERCENTAGE OF.CASE$ WITH INSTITUTIONAL ERROR 
BY TYPE OF ERROR AND ADMINISTRATIVE STRUCTURE 



Type of Institutional Error 



Type of 
Administrative Structure 

Independent Campus 

;;C!entral Office Reporting for 
Branch Campus System 

Campus-Level Reporting for 
Branch Campus System 



Overall 
Institutional^ 

^ ■ 30.0 
25.7 



Eligibility 



6.5 



IChi square = 12.*1, df = 2, £ = .002, for error/no error. 
2Chi square = 2A6, df = 2, £ = .2919, for error/no error. 
3Chi square = 22.35,.df - 2, £ = .0001, for error/no error. 



Disbursement^ 

25.8 
19.9 
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TABLE 3-17 



PERCENTAGE OF CASES WITHJNSTITUTIONAL Ekft&R BY 
RATIO OF PELL RECIPffiNTS TP UNDERGRAJJU^TE Er|ll|f 




Ratio of Pell Recipients to 
Undergraduate Enrollment 


Cases 
w/Errbr* 

(%) ■ 


*• 

Cases . 
w/Overaward 
(%) 


.j.^^/Uhd^raward 


0 '- 






11.5 


' #"'33;5 


.1000 -■ 


.1999 : 

i 


'36.1 


17.2 


' 18.9 


.2000 - 


.2999: 


, \^ 30.2 


l't.2 


15.9 


.3000 - 


.3999 


\ >32.7 


16.2 


16.5 


.4000 ' - 


.'f999 ;■ 


36.3 


11.6 ^ 


24.7 


.5000 + 




32.3 


*■ 


12.9 


Chi square 


f 

= 23.68, df = 5, £ = 


.0003, for errpr/no error; 
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TABLE 3-18 

PERCENTAGE OF CASES WITH ERROR BY INSTITUTIONAL VALIDATION 
FOR VARIOUS TYPES OF INSTITUTIONAL ERROR 



Institution Selects 
Cases for 
Own Validation 



Yes 
No 



Overall 
I nstitutional 1 

3*.0 

31A 



Type oi institutional Error 



Eligibility^ 

. 1^.9 
6.1 



Cost of 

Disbursement 3 Attendance'^ 



31.0 
26,8 



11. if 



Enrollmient 
Status^ ' 

22 i 6 

21.7 



Calculation/ 
Accounting6 

12.8. 

10.8 ■ 



CO 

I 

to 

00 



square - 1.7*^ df 
2Chi square - 1.74, df 
3Chi square - 4.93, df 
**Chi square - 5.28, df 
^Chi square - .23, df = 
6Chi square - 2.^2, df 



df = 1, £ = .1874, for error/no error. 
= 1, £ = .1875, for error/no error. 
= 1, £ = .0265, for error /no error. 



1, £ = .0216, for error/no error. 
, £ = .63 1 1, for er^^op/fK^rror.' 
1, £ = .1457, for error/no error. 
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TABLE 3-191 , 

PERCENTAGE OF CASES WITH VARIOUS TYPES 
OF ERROR BY PERSONNEL USED FOR AWARD CALCULATION 



Type of Error 

Disbursement^ 



Calculation/ 
Account ing2 



T ype of Personne l 
Professiohal Only 
Clerical Only 
Both of the Above 
Other 

Professional Only 
Clerical Only 
Both of the Above 
Other 



W/Error 



32.0 
30.0 
29.8 
18.9 

12.2 
13.6 
12.6 
9.5 



Cases 
W/Overaward 
(%) 

12.2 

10.1 

11.7 

7.0 

6.9 
9.3 
9.0 
5.^ 



Cases 
W/Underaward 
(%) 

19.8 

20.0 

18.1 

11.9 

5.3 
1^.3 
3.6 



iChi square = 2f.890, df = 3, £ = .0001, for error /no error. 
2Chi square = 3.299, df = 3, £ = .3't78, for error/no error. 



calculation/accounting error. For these two types of institutional error the likelihopd 
of making errors was fairly close whether " schools used professionals only, cleric^s 
only, or both, averaging about 30 percenf for disbursement error and'' about 11 percent 
for calculation/accounting error. THe "other" category had (;onsiderably lower* error 
rates for both types o^f error but the- fo percent ^ the cases attending schools 
responding to the "other" category and the range of responses^ falling into this category 
make speculation inappropriate and too risky. ^ ' . 

Another discretionary factor relating to the type of personnel used to perform 
award calculations is the method used to calculal^e awards. The ePror rates for schools 
using manual, automated, or combination calculation methods are displayed In Table 3- 
20 for disbursement, cost^of attendance, enrollment status and calculation/accounting 
efrors. The error rates were fairly sim^ilar for the three categories across all four 
types of institutional error. Only for disbursement,; error are^, there significant 
differences in error by method of award calculation, with 'automated' calculation* 
associated with the lowest error rate. * . 

Whether or not institutions reported routinely reve/if ying award calculations ' 
would be expected to result in improved quality. Table 3-21 shows the test for this 
hypothesis fpr disbursement error and calculation/accounting error. The differences in 
the results for both types of error are so small as to be statistically meaningless. 

A final discretionary characteristic dealing with award calculation is the 
frequency of award recalculation. We would expect that error rates would be lowest 
when awards are recalculated before each disbursement. Table 3-22 presents tests of 
this hypothesis for disbursement error and enrollment status error, respectively. \)(hile 
differences do exist, the statistical strength of these differences is fairly low. The 
findings are also somewhat c'ounterintui^ve in that the error rates were higher for'the 
"before each disbursement dPily" category. 

Table 3-23 pres^^s error rate5**for enrollment status error by whether the school 
reported checking enrollment sH^tus. While enrollment status error was lower for 
schools checking. enrollment status before each disbursement, as would be expected, 
the size of the difference in error rates is too small to have any statistical validity. 



3-30 



TABLE 3-20 
% ' 

PERCENT-AGE OF CASES WITH VARIOUS TYPES OF 
^ ERROR METflOD OF AWARD CALCULATION 



• Award 
Type of Error . Calculation 

Disbursement 1 • Manual . 

Autornated 

* ^ Combination 



Cases 
W/Error 

32.3 

27 !o 



Cases 
W/Ovferaward 
(%) 

13.2 

8.2^ 

7.6 



Cost of 
AtfendknceZ 



Enrollm^rtr 
Status3 



Calculation/ 
•Accountings 



Manual 

Automated 

Cdffibination 

Manual * 

■V 

AutoFT^ted ' 
Coml^inatiqp 
It' 

Manual 
Automated 



.1^ 



Combination 



10.9 
8.5 
^12.6 

,22.8 
21.3 
21.9 

12. II- 
*t3.3 



^Chi.square = 16.110, df = 2, jOeO^^for error/rjp error. 
2chi square = 5.35, d| = 2, £ s .0689, for error/no errorj 
3chi square = Jt^, df = 2, p = .69D6, for error/no errop. 
'^Chi square = .891, df = 2, £ = .6't05, foP error/no error." 



it. 8 
2.9 
6.7 

« 

8.1 

6.3 , 
3.8 

8.3 
8.1 



* 
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TABLE 3-21 

PERCENTAGE OF CASES WITH VARIOUS TYPES OF ERROR 
BY REPORTED ROUTINE REVERIFICATION OF AWARD CALCULATION 



Type of Error 
Disbursement! 



Calculation/ 
Account ing2 



Reverification 
of Award 
Calculation 

Yes 

No 

Yes 
No 



Cases 
W/Error 
(%) 

30A 

29.2 

12.1 
13.3 



Cases 
W/Overaward 
(%) ^ 

11.1 

12.6 

7.6 
6.7 



Cases 
W/Underaward 
(%) 

19.3 

16.^ 

1^.5 
6.6 



IChi square = M2, df = 1, 2 = .5260, for error /no error. 
2cfii square .73't, df.= 1, £ = .3917, for error/no error,. 



% ^ 
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TABLE 3-22 

PERCENTAGE OF CASES WITH VARIOUS TYPES OF ERROR 
BY FREQUENCY OF AWARD RECALCULATION 



Type of Error 
Disbursement! 



Frequency 
of Award 
Calculadkon 




Before Eac 
Disbursement Only 

When Enrollment 
Status Changes 
Only 

Both of the Above 
Other 



Cases 
W/Error 
(%) 



3^.6 

26.1 
29.6 
29.9 



Cases 
'W/Overaward 
(%) 



12.8 

8.2 
13.8 
11.7 



Cases 
W/Undera,ward 
(%) 



21. 



17.9 
15.8 

18.3.. 



Enrollment 
Status2 



Before Each 

Disbursement Only 26. 

When Enrollment 

Status Changes 

Only 19.0 

Both of the Above 21.9 

Other 22.0 



8.1 

7.5 
8.6 



18.2 
13. if 



IChi square = I7.5t^2, df = 3, £ = .0005, for error/no error. 
2chl square = 15.812, df = 3, £ = .0012, for error/no error. 



TABLE 3-23 



PERCENTAGE OF CASES WITH ENROLLMENT STATUS ERROR BY 
REPORTED CHECKING OF ENROLLMENT STATUS 



„ ^. \ 9^ses Cases Cases 

Reverification of . W/Error* W/Overaward W/Underaward 

Award Calculation (%) (%) 

' " ~ ' 



yes 21.8 

No 23.7 8.6 



15.3 
15.1 



* 



^Chi square = 1.720, df = 1, £ = .1897, for error/no error. 
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Table 3-2^ presents our finarbLvariate test of a discretionary characteristic: the. 
existence of a; relationship between ^disbursement method , and < disbursement error; 
Surprisingly, the ^rror rate was highest for schools paying 'by checik t)Ut requiring, the 
student to endorse the cHeck over to the institution. The vast*<TiajoVify 'Qf:scte6^^ used 
the fourth method-.-cr-editing the students' account Vand. disbursiAg ^ny%bal*ht^^ ^b^ 
check. Of course the^rrtethod o^ disbursement >yas h^aviJy^influenced by*the leVel 6f 
attendance, cost, and whether or not the aw^rd exceeded the amount the}stjyent ^ad . 
to remit to the institution. • ' . ^ . * 
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The bivariate findings on the. relationships between discr^|ionary institudohal 
^ characteristics and rates of error reveal few meaningful associations. Institutional 
validation beyond the high levels required' by thje Department of. Edu^ati not 
associated with reduced error.; Of the four discretionary, character isti<*s re^^^^ to 
award recalculation, only'metjnod of award calculation: yielded any mea^iigful and 
significant relationships, with disbursem;ent erroh ratbs lowe^'tor institulions using 
automated systems. The personj^el used for award calculafio; 
recalculation, and the routine verification of award, recalcula 
rates of error. Finally, institutions r^ifeporting procedures 
showed no significant reduction o£' related error and meth 
meaningfully related to disbursement error. 

There is <5ne additional institutional characteristic tha 
because it is a response to an >xternally Impose^ requirem 
validation reportfe^d. by each institution. Tal^e 3-^ shows, the 

reported burden of .Validation and cases with error^^t^first '^gla^ce, the resu^ appeai" 
to be counteHmuitive;/ j|^^ a burden ^f v;^li(Htion:had si^nificantiy ' 

less Qrror than 'those riof're"^brting . a burden.^' However, it is possible that t 
atutions reporting a iburden went to greater ef^rts in vartidatipn than those wt^p- 
t;4knd that'^1:l\fe burdeh paid.of f in reiduibed ,^rror. 




:y ^^^^^^ 
le af f ecf on 
Aliment status 
iement was^bt 



ily discretionary, 



lis is the burden o^.-. 
onsh|p between /the , 





(romD^ris^ris 



1 



A /t '^-ii'^he blyar||^|e dOm^risops fii the previous section looked at the rel^onships 
^',blftween eacH of^'^everalj^ftstiiuti and one*o^n(ore of 

^^^h^^.instituti^^^^ err^jt mlafsures. Thtis represented a first effort to ici^r^p some 
; possible feausa^vs for error. 



While t|n^ results of these' bivariate, analyses ■ iwa.^ 
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TABLE 3-24 



PERCENTAGE^OF CASES WITH DISBURSEMENiMR^OE^ 
tYPE OF DISBURSEMENT MADE 



r Disbursement Method 

,v 1. pieck to^^^ for full 

. amount (only) 

2. Check to Student-Endorsed 

over : to institution (only) 

3. Credii/tP Students 

acQ3i\iht;(only) i 

Credit to Account— Balati^e 



* /by^Qj^ck (pn^yX v^^^ 
Both^if and 1 ' ' ..^^ 
^h >an3^ ' 

■. V.'.- ^ \ " . : : ' ' ' 

viAoy other 'tonfibination 



Cases 
W/Error 
(%) 



35.4 
44.5 
22.6 

:32.4 
'l8.9 
25.0 
28.9 



dases 

^/^«^ward: ; W/Unttei^award 




• .1: 
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' TABLE 3-25 

PERCENTAGE OF CASES WITH INSTITUTIONAL ERROR 
BY REPORTED BURDEN OF VALIDATION 

Cases Cases Cases 

R,..^«D W/Error* Wveraward W/Underaward 

Burden Rep orted (%) (%) (%) 



Yes ' 30.7 iij.g 



15.8 
23.2 



*Chi square = 35.37, df = 1, g = .0001, for error/no error. 

y ' - ■■ . ■ ' , 

. u 
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have been useful, a second step is needed. Multivariate analysis permits the joint 
testing of the effects of several institutional characteristics,, the independent 
variables, on error. In theory this makes i't^ possible to explain a greater proportion of 
the variance contributing to institutional error, assuming that the model chosen is 
appropriate. We have elected a linear multiple regression model with institutional 
error as the dependent variable. 

Based on the results of ouc bivariate analyses, which revealed institutional 
differences in rates of error by type and control of institution, we are conducting 
separate regression analyses for each type and control of institution (with one 
exception) and one combining all institutions. Because of the small numbfer of private 
2-year institutions, these were combined. with proprietary 2«year institutions. Thus, 
we estimated six models, one for each of the following groups: 

• All institutions' 

• Public 2-year institutions 
' < Public ^-year institutions 

• Private ^-year institutions .. 

• Proprietary under 2-year institutions 

• Proprietary or Private 2-year institutions. 

X 

The institutional characteristics which were jointly considered in the model as 
independent variables were: - 

Method of disbursement 

Frequency of disbursement • ' 

Payment calculation method ' 
Institutional validation 
Recalculation of expected disbursement 

Routine reverificatidn of calculations , ^ 

Personnel calculating awards ' ~ 

j.1 • - ■ 

Whether enrollment status was checi^ti before, computing award 

, 78 ' 



Whether 1982-83 validation caused a burden. 

The model related net institutionial error for the sample of students falling in^ each of 
the categories to the above characteristics. , - ' . 

Table 3-26 presents the results of the regression models, with the type and 
control of institution in the model indicated in the . column headings. All models share 
the same nine independent variables, shown a's^^w entries. The last row shows the R- 
squares for each of the six models. This can be interpreted as the proportion of 
variance in the dependent variable (institutional error) explained by or accounted for 
by the independent variables included in the model. The maximum possible value of R- 
square is 1.00. For an exploratory analysis, an R-square of .150 (15 percent of the 
variance accounted for), as fouftd for private 't-y ear schools, is considered quite 
strong. Notice that four of the models accounted for fairly large proportions of the 
variance. The R-squares for all of the six models are statistically significant. 

The ^^st row^.of the table, labeled intercept," is usefdl in interpreting the 
remainder oiHhe table. This value is the average error which would be expected for 
students attending schools with particular institutional procedures. These procedures, 
shown as base responses on, Table 3-26, are as follows:' 

• Method of Disbursement— "other combinations" 

• Frequency of Disbursement— ^Varies by program" 

• Payment Calculation Method— "manual" ^ ^ 

• Institutional Validation— "yes" 

• Recalculate Expected Disbursement— "when enrollment changes" 

• Routine Reverification of Calculation— "yes" 

• Personnel Calculating Awards— "professional" 

• Check Enrollment Status Before Computing Award5~"yes" . 

• Did 1982-83 Validation Cause a Burden— "yes." 



TABLE 3-26 



ESTIMATED DIFFERENCES IN AVERAGE NET INSTITUTIONAL ERROR 
ATTRIBUTABLE TO DIFFERENT INSTITUTIONAL PROCESSES AND METHODS 





All . 
Institutions 


Public, 
2 Year 


Public, 
■ Year 


Private, 
* Year 


Proprietary 
Under 2 Years 


rTopriexary 
or Private, 
2 Year 


Intercept ^ 

■ ;.v " • ■ ■■ ■ ^r. 


-69.2 


; 37.7 


-68.3 


-55.5 


^8^.2 


136.3 


Method of Disbursement 


• ' ; ■ 


* 




« 


* \ 




1. Check to Student, Full Amount, Only 

2. Check to Student/Endorsed Over to 

School, Only 

3. Credit Account, Only 

^. Credit Account and Disburse Balance 
by Check,- bnly * 

5. Methods 1 and 2 i 

6. Methods 3 and * ^ C 

7. Other Combinations 


-6't.l* 

-lO't.6 
-22 A 

-18.0 
-8.0 
12.7 
Base 


-115.3 

' N/A 
N/A 

1.0 
9.7 
9.8 

Base 


32.8 

29.1 
N/A • 

10.1 «■ 

-6.0 
,1^4.3 
Base 


N/A ' 

-265.1 
-102.3 

-85.2 
-34'. 8 
-76. 

Base , 


. N/A 

10^1 
19.9 

21^5.6 
1^96. S 

362. ' * 
Base ' 


-1019.5 . 

-555.6 
-717.6 • 

N/A . 
-393.9 
Base 


■ ic\|udicy iix L^ioLiuroeiiidi L 


It 


■)(■ ' 

-195.8 
-32.8 
-19.8 
60.1 
oase 










1.. Monthly . . 

2. Bimonthly 

3. Quarterly or 3-^ Times per Year 
Once, per Term 


-92.1 
33.3 
61^.9 
57.5 . 

'. ^ oase 


210.3 • 

N/A 
55.0 
38.2 

oase 


330 .-9 
-108.7 
-23,2 
-28.1 
oase 


-238 0 
N/A' ' 

-1^96.2 
- 75.2 
. > Base . 


-115 ^ 
N/A 

Base 
550.7 
N/A 


Payment Calculation Method 




* 


■ ^ ^'V 








1. Manual * 

2. Computerized 

3. Combination 


, Base 
-5.9 
: -3.7 


Base 

-97.1 

-12.1 


Base ' 


Base 
-^2.1 
-28.3 


Base 
: 251^.2 
2W.7 


Base 
-601^.6 . 
-133.8 . 


Institution Validation 




■ . 


■ P ■ ■ 


■ . ■ * " ■ 






i.'Yes - . . 


Base 


Base 


Base ^ ' 


Base 


Base 


Base 



2. No 



-5.1 



6'f.5 



12.0 



-59.8 



286.2 



-357.9 



♦Indicates that the characteristic is a statistically significant 
r'--^ 'n the column heading. / 

FRir ' 



.05) predictor of institutionaF^erroV for the typ^p^'^ttntrol of institution 



TABLE 3-26 (Continued) 



ESTIMATED DIFFERENCES IN AVERAGE NET INSTITUTIONAL ERROR 
ATTRIBUTABLE TO DIFFERENT INSTITUTIONAL PROCESSES AND METHODS 



I.' • 


All 

Institutions 


Public, 
2 Year 


Public, ^ 
k Year 


Private, 
* Year 


Proprietary 
Under 2 YeariT 


.proprietary 
or Private, 
2 Year 


Recalculate Expected Disbursement 






« 


» 




» 


1. Before each Disbursement 

2. When Enrollment Changes 

3. Both 
^. Other 


29.6 
Base 
0.7 

20.^ 


20.7 
Base 
^1.1 
32.3 


-^♦5.5 
Base 
-29.7 
- 1 5 9 


9.2 
Base 
-9.6 

Of • t 


58.6^ 


574.8 
Base 
241.6 


Routine Reverification of Calculation 

1. Yes 

2. No 


Base 
- 19.0 


Base 
2.0 


Base 
37.6 


Base 
17.1 




♦ 

Base 
-643.9 


Personnel Calculating Awi^/ds 




odse 

- i - ... 

-20, 
34.6*- 


# 


0 




* < 


1 Pr^^ f ^^^^ 1 r\r^ A 1 

2. Clerical 

3. Both 
^. Other 


Base 
17.2 
16.8 
12.8 


Dase 
40.0 
.33.7 
-'58.8 


Base 
35.3 
75.7 
31.9 


Dase 
N/A ' 
»J/A 
N/A 


Base 
368.7 
-728.2 
N/A 


Check Enrollment Status Before ^-.^ 
Computing Award 




■ # 




* 




i ■ 

* 


1. Yes ^ 

2. No 


Base 
-0.2 


Base 
-90.9 


Base 
23.9 


Base 
76.9 


Base 
N/A 


Base 
363.1 


Did 1982 83 Validation Cause 
A Burden? 




» 






» 


* 


1. Yes 

2. No 


Base * 
3.5 


Base 
-65.7 


Base 
24.1 


Base 
112 


Base 
-255.3 


Base 
350.3 


Proportion of Explained Variance (R-squared) 


.0^7 


.195 


.032 L 


.150 


.535 


.358 



* Indicates that the characteristic is a statistically significant (]^ .05) predictor of institutional error for the type and control of institution 
shown in the column heading. 
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The numerical entries in tlie table represent the differences in average net error 
attributable to being in those categories of the variable rather than falling into the 
base categories. For example, the "all-institutions" model (column one) indicates that 
students attending schools where a check to the student for the full amount was the 
only method of disbursement (response //I) had average errors which were $6^ lower 
than students attending institutions , which used the base category, "other 
. combinations" (response //7). Thus', the average net error for this- group of students 
was $ -133 ($ -61^ plus the intercept of $ -69), assuming that the base category applied 
for each of the other eight variables. Using the information in this table, the direction 
and amount of change it\ error can be derived for numerous combinations of response 
categories.^ It should be understood that th« results would change if different base 
categories were used, but the 'relationships between the response categories for a 
given variable would remain the same. 

Consider another example of this table using thfe second variable, frequency of 
disbursement. Students being paid^ "once per tgrm" (response M) had average net 
errors which were $58 higher than for students attending schools where frequency of 
• disbursement varied by program (the base), which worked out to an average net error 
of $ -1 1 ($5.8 less error than the base of $ -69). 

Asterisks in the rows containing the names of the variables indicate that the 
variable has a statistically significant relationship to net institutional error for 
students attending schools in the group indicated by the column heading. 

The intercept row reveals the importance of type and control as an intrinsic 
explanation for institutional error. The entry in column one for all institutions 
indicates- that the average net institutional error for students falling into the base 
categories for all varia'bles was $ -69. This value exhibited considerable differences 
for the different subgroups of institutions, ranging from a positive $136 for 2-year 
proprietary or private institutions to an average net underaward of $8't. for proprietary 
schools with programs under 2 years. This finding is consistent with the earlier bi- 
, variate analysis. 

Method of disjaursement is statistically important in five of the six type and 
control categories. The fact that the categorical effects of this variable are 
important is somewhat muted by the fact that the differences attributable to being 

ERIC SA ^ 



in one category are not consistent across the models. For example, the estimates for 
.the; category "credit atcount, only" (response #3) ranged from about $20 to a negative 
-f?^?* This is consis^tent with the bivariate results which were somewhat unclear. 

• Frequene:^,-^),f, disbursement was important in four of the six models. However, 
the results y^^|^£(; widely across models with estimates for any one payment frequency 
represenling^pCtjitiVe and negative differences in average net error from the base 



category. '^.^/^ 



Method of payment calculation was only significant once and the estimates were 
not stable across the six models, with computerized systems having average net errors 
per student from $605 lower to $25^ higher than manual systems. 

The results for the remaining six variables were ' similarly mixed, with some 
variables being statistically important for some type and control groups and not for 
other type and control groups. The signs of the estimated differences were both 
positive and negative for every category of these six variables with one exception: the 
clerical category for type of personnel used to calculate awards relative to use of 
professionals as the base. Here, use of clerical personnel to calculate awards resulted 
in higher errors across all institutional types. 

3A CONCLUSIONS 

A- 

I 

Institutional error accounted for about 30 percent of the net cost of Pell 
program error. It was about equally likely to be an overaward or underaward, but 
dollar values of overawards exceeded dollar values of underawards by nearly two to 
one. Major recommendations regarding efforts to reduce institutional error include 
the following: 

• . Most of the overawards were due to one procedural eligibility requirement, 

Jack of a Financial Aid Transcript (FAT) for transfer students. With the 
exclusion of this error and two other minor procedural errorSv net 
^ institutional error was a modest underaward. A change in the requirement 
for collecting the FAT, which c^|have no bearing on the award in any 
case, wo4ld cut approximately 28 percent of all institutional error. ' 

• Other major contributions to institutional error were cost of attendance 
: and enrollment status. These errors were due partly to confusion about the 



institution's responsibility for tracking ^inor '(*a'nges across the academic 
year. Clarification of these responsibilities and^<irnmunication of them tog 
institutions would si^ificantly reduce error. ■'^^^ 

On the whole,' disbursement errors, including cost of attendance and 
enrollment status'^ were underawards. This suggests caution by institutions 
and their awareness of their own liability, but occurs somewhat at the 
expense of student entitlements. 

Efforts to isolate the possible causes of error l&d to the conclusion that 
factors intrinsic to the institution, such as. type and control, were related 
to error but that discretionary institutional proc^l^ur'es, Vhich could be 
changed, were not strongly related to error. ^ 

Proprietary institutions, which are generally clock-hour 'institutions, (had 
.the highest rates of error. This may be relat«d to the greater complexity 
of award calculation for clock-hour institutions. Fouf year instjiutions had 
the lowest rates of error. ' * ' M 



CHAPTER * 
STUDENT ERROR 



This chapter presents estimates oi student error 
1982-83. Total "student error overviewed in Chapt 
component parts as a means of ^attempting to identify th 
anj;! regulations that might be effective^^^. reducing 
Significant findings pre^nfed in this chapter include: 



The 1 982-83 ^lii; Grant program experienced^an;^|tk^ 

riet student error, or $86 per recipient^ • -'^-'Ir^ < ^ -V 

Depenjfency status error was tHl^l^rg^st single If o?^m.^ student error',/ ^-^ '^^^^^^ 

comprising approximately $6^ milUpn in payment tons<^ " ^ 




The top six forms of student- 6^or ^wi^r* not . directly y€|rm^b1[€^.thr^^ 
1982-83 validation prbcedures. T ' ^^z ";^ f ^Vf S ' v^-* 



Extensive bivariate analysis incjic^ted^l*^^ . 
enrollment were the only variaBiesvjWiw f^^ correlatioit- to . 



student error. 



my 



Sidicatedh^^ students ffcJwt higher 



Multivariate anal: 
incomes, 
have 

^Correlation of studfe^t error *and institu^^f^Tf^ type ' -and c^rol w?is 

> V, V 



les, smaller fan^l|^;^|jad old^j^^^^^ to .-^^^ 

overawards pf Pe^unds du^ t^misreporting of applfcation data. ' 



confirmed by the multivariate analysis. 



if 



Use of actual tax return data improved the accuracy of adjusted grqss^f^l^ 
income and U.S. taxes paid, but ;had littl^^^^^^ affept (J^ QtHer .^\v^^;^ 

application items. " , ' " ' 



NATURE AND TYPES OF STUDENT ERROR 



-In Chapter 2, student error was defined as the difference between the correct 
award the ^tudent should have received and. the award calculated using the appliciti 
data submitted by the student and the correct cost of atteijdance and enrolls 
status. Any errors which might have been made by the institution are factored 

9i ' * /\ 




EKLC 



^-1 



8 



the determination "of student error. " Ther^re, studepi error definitions are 
contingent on definitions of institutional error. ^A^^-a consequence, estimates of 
student error differ according to the treatment oi#tudents-^'ith massing Statemehfs of 
Educational Purpose (SEPs), Financial Aid Traijiscc>fs '(FaI's); or invalid Student ' Aid- , 
Reports (SARs). For example, if students with SEP/FAT/invalid SAR error are tceatecj 
as ineligible, any student errors made by this subset o^f students'would be^voided by our\ . 
formula, resulting in slightly lower estimates Of ' student*error.^ However, it c^n be 
assorted that , students with these, institutional ^terrors sl^uld be" included^ \n student 
error in order to give , a more representative, vj^ew ^of stildent pirror. TT\^ can be 
accomplished by treating SEP/FAT/Invalid SAR errori as eiigfole and yields/^ghtlyX 
higher estimates .of student error. Both of these- estimates' afe'Vpresented in'par?»ilel 
wherever it was felt the analysis required du^l^d^tmeniiyas was^one th^-ougfrebt; 
- ehapters 2 and' 3.^ : W^^^ only one set of figure^, i^,j9shp«^^ aft .s^ud^hh witli ,^ 
■SEP/FAT/Invalid SAjl errors were considered as eliglblo, qr|ess oliej|ji'se.notici. '-' ^' 

Studeht.errpr may occur whenever there are one or morj^n<^rr||k^resp^drt^^ 
the appUcation questions. (Not all discrepancies in' apiliiCatSn item'^'have-^yment 
consequehc.es. Some discrepanties, for example, are* to'b I'hiairto'' affect the SAl 
sufficiently to induce payment change. Other discrepanc^ssv/Tiaiiicha^e the SAI, but 
not the award because of the role that cost of attendance plays^ a^gd calgulafion. ' 
Discrepancies In asset or debt items may not change the a because a certain {[^ 
amount of equity is protected in the award calculation^. All dis<yepaij||ies are 
separately discussed in this chapter and are always referred t^s dfiferji^ancies father : 
than errors to avoid confusion.) Student error estimates pre^s^^nted^ ^-in thi^- fefeport 
represent the payment consequences for any and ail incprrect responses to appiication 
questions. In this chapter we decompose student error by ^tim^ingt theo,;^llar 
payment consequences of error for most of the application items. -Th^e estimliJes are^ 
.usi^aiiy for a singie appiication item, suqh as adjusted gross incon^e/ f^seho id size, 
.^tc. However, there are some application items which are best considered togetjjen 
such, as home vaiue and home debt (which become hom^ equity), gr -the six iter 
determining dependency status;.^ / 
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As noted in Chapter 1, the findings are presented in two primary dimensions: . 

• The percentage of cases with errors th^t result in payment consequences in 
excess of $2 wj - 



V t^, ' ^ J^-' 'I'he payment consequences of the identified errors 
v - n.^^^ - in absolute terms (sum of all dollar errors) 

i ■ Da . ) ' i" terms (overpayments minus underpayments). 

v^-. -vv 'v''/ ■ . V ■ = . ■ ■ 

* /. j?' Jni-a^^^ mean error per recipient and mean error per recipient with error are 
^'f^!^"*^ I"^"^"^^^' used to illustrate the payment consequences to individual students on 
average. Mean error per recipient, is the sum of. student errors divided by the number 
including those with zero student error. Mean error per recipient with 
^ divides the sum of error by the number of students with either overaward or 
I*, underaward, excluding students with zero error. Payment error as a percentage of 
dollars awarded is also cited to show the relative amount of error. 



^ : ' 4,2 STUDENT ERROR FINDINGS 



iv-v Student error is reviewed at three levels, The first two, overall error and 

individual components of error show payment consequences of error. The third, 
I discrepancies in application items, illustrates prevalence of error without regard to 

:^ payment consequences. 



4.2.1 Overall Student Error 



Table 4i-l summarizes the amounts and rates of student error where SEP/FAT/in- 
valid SAR errors cause cases with those errors to be excluded as ineligible applicants. 
The program-wide dollar estimate of absolute student error was $328 million, 
comprised of $272 million in overawarded cases and $55 million in underawarded 
recipients, for a net student error of $217 million. That amount of error averaged 
$129 per recipient in absolute terms and $86 in net terms. 

For recipients with error, the meaawas $328 in absolute terms and $217 oh a net 
basis.t Overall, nearly 40 percent of all cases were projected to have some form of 
error with a payment consequence ir^ excess of . $2, of which about 31 percent were 
over^wards and 9 percent were underawards. Absolute error as a percentage of dollars 
awarded was 14 percent. Overaward was il peVcent--of the total awarded dollars, 
while underaward was 2 percent of the total. 

4r 
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TABLE 



t 



I. H^' 



) . 

AMOUNTS AND RATES OF STUDENT ERROR 
TREATING THOSE WITH SEP/FWiNVALID SAR ERRORS AS INELIGIBLE 



Absolute 
Student 
Er^or 



Net 
Student 
Error 



Student Student 
Overaward Underaward 



Program-Wide Estimate ($ Millions) 



328 



217 



272 



•55 



Mean Error per Recipient ($) - 



129 



86 



lOS 



•22 



Mean Error per Recipient with 
Error ($) 



328 



217 



351 



-2f9 



Percentage of Gases with Error 
Greater Than $2 



39A % 



39A % 30.6 % 



8.8 % 



Error as ajfercentage of Dollars 
Awarded^ 



11^ % 



9 % 11 % 



2 % 



/ 



r 



4 
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When cases with SEP/FAT/invaUd SAR errors were treated as eUgible, the 
amounts and rates of student error increased slightly, by the amdUnt'^bf student error 
in those cases otherwise excluded. Table ^-2 shows the magnitude of |htis additional 
^ percentage of dollars awarded, the increase is negligibi^;' ' v 

/ The range of student error is shown in Table *-3v both with and without 
SEP/FAT/invalid SAR errors included. More than 60 percent of the cases were within 
" $2 of the correct award when application data were.usedv None of the ranges in the 
negative half of the scale had a very high proportion of the cases; underawards were 
spread fairly evenly among the ranges. At the other end of the scale, however, there 
were larger proportions of cases withVror. The primary significance of the more 
than 8 percent in the highest ran^e is that it included most cases where the larger 
application-based awards proved to be entirely Otn error (reduced to 0) when correct 
data were used. 

*.2.2 The Gomponents of Student Error 



While the overaU\l^scription of student error is useful in identifying the 
magnitude of error that is taking place, it does little to ilerftify potential causes of 
that error so that corrective measures can be developed to reduce their incidence. In 
order to identify the primary causes, it is necessary to decompose student error and 
determine those elements that are most troublesome, most amenable to corrective 
measures, and most likely to yield cost-effective results. . ^ 

' The decompositio^of student error into the* likely dollar effects contributed by 
each application item is only an estimate because it dpes not take into account the 
interactions between items. For example, in an actual case an underreporting of 
a|j^Sted gross income (AGI). might be totally offset by an equal overreporting of other 
income and benefits, yielding no net payment consequence, assuming there were no 
other applicatioft item discrepancies. In estimating the separate payment 
consequences of each application item for this example, there would be net overaward 
error for AGI and net underaward error for other income and benefits. Therefore, 4n 
interpreting the amounts of rietistudent errors due to individual items caution must be 
used. . • 



TABLE 4-2 

AMOUNTS AND RATES OF STUDENT ERROR 
TREATING THOSE WITH SEP/FAT/INVALID SAR ERRORS AS ELIGIBLE 



Absolute 1 Nist 

Student Student Student ° Student 

^rror Error Overaward Underaward 



Program-Wide Estimate ($ Millions) 



31^5 



231 



288 



-57 



Mean Error per Recipient ($) 



137 



91 



llif 



-23 



Mean Error per Recipient with 
Error ($) 

Percentage of Cases with Error 
Greater Than $2 



332 



222 



355 



^lA % 'iH.l % 32.0 % 



-250 



9.1 % 



Error as a Percentage of Dollars 
Awtirded 



11^ % 



10 % 



12 % 



2 % 



\ 
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4-6 



TABLE-*-3 

PERCENTAGE OF CASfiS WITH STUDENT ERROR BY DOLLAR RANGE 



Student Error Treating 
SEP/FAT/Irtyalid SAR Cases as Ineligible 

8.2 
7.0 

2.5 
3. if. 
3.8 
1.2 

0.3 
1.2 
1.9 
0.8 ^ 
1.6 
1.6 



Student Error Treating 
SEP/FAT/In valid SAR Cases as Eligibl 
■ (%) 



8.7 

7.2 

if. 8 

2.6 
.3.6 

3.8 

1.3 - 
58.9 

Q.3 

1.2 

1.9 - 
, 0.8 
1.6 
1.7 
l.if 
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Table k-k displays the distribution of error in its component par'tsi most of which 
is at the application-item level. A few items are composites of two, or mote 
application items. These are dependency status (from the six dependency statUs 
questions), and home equity, business/farm equity, and real estate/investment equity 
, (the difference between value, and debt for each .of the items). With the exceptiorj of 
/\AGI, the reason for which is footnoted, the components are listed in descending rank 
order of their net payment consequence or error. The mean net error per .recipient in 
the first column mirrors the progra>n-wide' estimates in the third column in terms of 
descending magnitudes. 

There are three factors that affect the values reported for net error per 
recipient in column one Of Table i^-i^. First is the frequency of occurrence of the item. 
For example,, most cas6s have AGI while few cases havte elementary/secondary tuition 
deductions. Second is the frequency of error in the items. Finally, the dollar 
consequences or the size of the typical error will influence the average error. ° 
t " ■ ■ ■ ■ , ° 

- % , Dependency status error was th€ highest ranked error, averaging over $25 per 
V recipient and $t^65 per recipient with error for a to|al of over $6't milUon. The second 
most costly type of i^n^ation error involved th<^ jhisreporting of other"nontaxable 
income, costing the progra^ about; $'t6 million in ,1982-83 and averaging over $19 per 
"recipient and $191 per reci^'ent with error. Hjusehold size Ranked third, .with error 
averaging nearly $14 per recipient arjjl $140 pe^ecipient with error for a total of.$3* 
million. i ' ' ' 

'■■ ■ ■/: . ^/' 'V ' ' ■ - 

The second column ^shdws th)9 , mean size ^of the . error ;f or, only those cases in 
error. On that ba^is, the rahk-oi^r sequence;:-of^ most ^rg^ er'rdr items changisd 
■ somewhat: The errdr in sjtvtd^nt rSt^^ status affected the mean award size by -$903, 
denoting a sigbificaht. (i|iderawarding prcfele^ for tHdse cases whepU^ occurred, even 
' though it hajspened veby infrequently. Business/farm equity was the nShs. most" seriqus - 
error when it. occurred (although- infrequently), affe^irig tjie yiean awlrd\y %t^7H per J 
, ' case with ertor. iep^ndency status error occjupied \ird '^e, with a mJan average - 
error of $'f6'5 per case, followed by AFDC/'^Fffi '^^if^V'pfx^^case error atid 

depenaent student»|/sj)6use's income at $331 per c||^ with error. The magnitude then 
feir to. -^27*_ for^ sturfents' SociaL Se<^urity educ^||nal benefits (an error fjiat SyJill 
' disappear .'in ai^ther year , because of ^rogj-am ch^anM|)i $255 for real estate/iriijfei^^ 
meht equity, and. $191 for other hbhtaxaible income. W^^^^^^ . > . . ' 
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TABLE Ih4 ^ 
STUDENT ERROR BY APPLICATION ITEMS \ 



Application Item 



Adjusted Gt^gss Ineome/Dependency Status' 

Adjusted Gross Income 

Dependency Status^ 
Other Nontaxable Income 
Household Size ^ 
Nun>ber in Postsecondary Education 
Home Equity 

Dependent Student's (and Spouse's) Assets 
Dependent Student's (and Spouse's) Income- 
U,S, Taxes Paid 

Spouse's/Mo tiler's Earned Income 
Real Estate/Investment Equity V 
AFDC/ADC , 
VA Educational Benefits » 
Marital Status (Parent) 
Business/Farm Equity ^ 
Gash/Savings/Checking 
Medical/Dental Expenses .* 
Applicant's/Father's EarnotHncome 
Elementary/Secondary Tuition 
Marital Status (Student) ' 
Social Security Benefitt (Parent) 
Social Security Educational Benefits 



J 



Mean Net 
Award Error 
Error Per 
Recipient 


Mean Net 
Error Per 
Recipient 
With Error 




Net 
Program-Wide 
Estimate 




, . ($) 






(S Millions) 


Rank 


6.70 


V 16<#,70 




•80 
16 


_ 
7 


25.30 


465.00 




, 64 ' 


. 1 


19.20 


191. 3j0 




46 ; 


2 


14.10 


139.80 




34 : . 


3 


9.60 


162,60. 




23 ! 


4 


7^0 


U3.10 






5 


7.io; 


39.20 




^ 17 






'330.80 




■'7;^^"l2' 


' 8 


1. 00 


29.00^ 




2 


9 


.90 


91.40 




.'■ ' 2. 


10 


.70 


. ' 255.40 


V ■■ 2 ■ 




.70 


402.80 - 






li 




69: 10 




1 ' V ■■ 


V3 


M ■ 


132.00 






1<» 


.30 


• ; 473.80 




' "1 ■ 


15 


\30 V 


' -35.90 




/ ^ 


16 




11.80 


J 


. +0 / 


17 


'.Hi I 


-5.40 ' 


-0 


18 




-44.00 






19^ 


-.30 


' -903.00 




'':] 


20 


-1.20 


-82.60 , 






21 


-7.10 


-274.20 ' 




-17 


22 



In 1980.- 1981 Adjusted Gross Income/Dependency Status Error were presented together as Adjusted Gross^jcohie (AGI) Error with an 
exp anatory footnote. The rationale for this was that Dependency Status Error represented^use of the incorrect AGI and other 
application values. In the interest of a greater clarity, we are presenting AGI Error and Dependency Status Error separately. si^KTc 
they are, in ^f^^^rrors that represent different problems. For purposes of comparison, We have ^hown the combined AGI/Depe^dcncy 
status error lor l7o2 —1983. * ■ 

Dependency status error is computed in the same way as Overall student error (an approach which is more accurate but cannot be used' 
with individual application items). The figure reported here treats dependency status switchers with SEP/FAT/Invalid SAR error as 
ineligible. If they were considered eligible, dependency status switcher error WQuld be $70 m|llipn. 
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*.2.3 Discrepancies in Application Items 

Another way to look at the distribution of student error at the application-item 
lever is by dollar range of discrepancies. Table ^-5^ shows that distribution of 
discrepancies between verified values and application values for selected items for 
'dependent students-. Not all discrepancies- resulted Id payment consequence's and some 
of these discrepancies were.-well within ED^s validation-tolerance levels. Clearly those 
items that were validated, AG! and U.S.' taxes paid, had far less error than non-, 
validated items. About' 85 percent of the dependent student cases were within $2 of 
actual or "best" values on these items and more than 90 percent were within the 
tolerance ievel of $300. The accuracy dropped off dramatically for the non-validated 
items, although 88 percent of the reporte(J*^s^lues foe Social Security educational 
-benefits, 90 percent of the reported ^values for other nontaxable income, 8^ percent 
for cash, savings, and checking, 74 percent for^dependent student income, 95^percent 
for dependent student assets and 78 percent of parental Social Security income reports 
were within $500 of the correct values. / 

Table 4-6 displays similar data for independent. recipients. Here the difference 
between validated and non-validated data was not so clearly apparent. U.S. taxes paid 
was as accurate as it was for dependent students. The accuracy of adjusted gross 
income fell off a, bit, perhaps from confusion ak to what was to be reported, i.e., 
student aid, gifts, etc. The percentage of cases with other nontaxable income within 
$2 was greater f6r independent students, but there was still a good number of cases 
(16.5%) with error in excess of $500, which in an independent case can make a sizeable 
difference in the award. Cash, savings, and checking was not a problem item, as 80 
percent of the cases were within $50 of the correct value and less than 5 percent were 
more than $500 discrepant. Mlsreporting of Social Security educational benefits was a 
problenTipr 1982-83, but will disappear as the phase-out of that program is completed. 
Half (n the cases with VA educational benefits had error, nearly all of it greater than 
$500, making this discrepancy an important source of error for those few students 
eligible for VA benefits. 

• Table 4-7 presents sohie additional information on the rates and amounts of 
discrepancies between best values and application values for 13 selected high error 
items. This table provides separate summaries for dependent students (on all 13 items) 
and independent students (on the 10 applicable items), excluding those recipients 
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TABLE ^5 



PERCENTAGE OF DEPENDENT STUDENTS WITH DISCREPANOES 
BY DOLLAR RANGE FOR SELECTED APPLICATION ITEMSl 



, Value of Item 
Discrepancy ($) 



iO,OOi 
^,001 
2,001 
1,001 
' 501 
^ 301 
*^^201 
101 
51 
3 
-2 
; -3 
-51 
-101 
-201 
-301 
-501 
-1,001 
-2,001 

-5,oai 

-10,001 



and 
to 
to 
to* 
to 
to 
to 
to 
to 
to 
to 
to 
to 
to 
to 
to 
to 
to 
to 
to 

and less 



more 
10,000 
5,£00 
2,000 
1,000 
500 
300 
200 



io6\^' 

200 



-300 
-500 
-1,000 
-2,000 
-5,000 
-10,000 



Gross 
Income 


Taxes 

Paid 


social 
Security 
Educational 
Benefits^ • 


Other 
Nontaxable 
lncome3 


Cash/ 
Savings 
Checking 


Dependent 
Students 
Assets 


Dependent 
Students 
Income 


jQCtal 

Security 
BenefiU 
(Parenty2 


i.l 


0 


0 


.1 


.5 


.1 


.1 






0 


• 5 




.5 


.1 


1.1 




1.5 


.3 


\5 


. 2.8 


2.0 ^ 


.2 




2.9 


.6 




3.1 


3.0" 


2.8 


1.1 


5.7 


;i.8 


.8 


.8 


<».2 


2.8 


^ 3.0 


2.9 


7.V 


/2.5 


.5 


1.0 


2.6 


7.7 ■, 




3.5 


• 5.2 


/ 




.6 


3*7 , 


2.3 


2.6 


3.2 




/ 


.6 


.8 


3.7 


6,1 


3.9 


6.0 


V.7 


: 1-1 




.8 


1.0 


3.<» 


3.9 


<».6 


2.7 


/ 2.9 


1.8 


.1.8 


^ .5 


6.1 


13.3 


21.2 


5.3 


2:5 


•85.0 


8<».2 


69.6 


. . 6<t.8 


<*9.7 






58.5 


.8 


1.4 


3.^ 


0 


2.? 


<».5 


2.5 


1.8 


.6 


1.0 


.5 


0 


1.5 


2.<^ 


K3 


1.1 


.3 . 


1.6 


.5 


0 




,2.7 


1.2 


3.6 


V .1 


1.1 


.5 


0 


.8 


l.<> 


1.2 




.<* 


1.3 


1.0 


0 


1.2 


1-9 


' 1.5 


3.2 


*7 


1.5 


2.1 


0 


1.7 


1.9 


2.6 


3,6 


.6 


.8 


1.6 


* 0 


1.6 


1.1 


1.9 


*.3 


1.0 


.5 


0 * ' 


0 ^ 


i.5 


■2 


1.9 


<^.0 


.8 ' 


0 


.5 


0 


1.5 


0 . 


.5 


2.2 




.1 


0 


0 • 


l.p 


0 


.2 


' 0 



The number oi dependent students with nonzero VA Educational Benefits was 5, too small for meaningful analysis of discrepancies. 
Includes only cases with nonzero values. . * ' ' 

Because the central processor for Pell applications provides only a total amount for other nontaxabfe' irK:ome we could not use the application 
value as a default in our verification of the separate contributing parts. Thus, we accepted undocumented values for the separate parts. To avoid 
the problem of failure to report any of the separate parts during our documentation efforts, we set the amount reported on the application as a 
minimum for the total value. This tends to overestimate other nontaxable income somewhat and precludes negative discrepancies. 



97 



o 

ERIC 



TABLE 

PERCENTAGE OF INDEPENDENT STUDENTS WITH DISCREPANCIES 
BY DOLLAR RANGE FOR SELECTED APPLICATION ITEMS 



» Value of Item 
Discrepancy ($) 


Adjusted 
Gross 
Income 


U.S. 

Taxes 
Paid 


Social 
Security 
Educational 
Benefits 1 


, Other - 
Nontaxable 
Income^ 


• Cash/ 
Savings 
Checking 


VA 

Educational 
Benefits! ' 


10,001 


and 


more' 


..2 


0 


0 


0 


0 


0 


5,001 


to 


10,000 > 




0 


0 


.3 


0 


0 


2,001 


to 


^{000 ' 


1 •/ 


• 2 


1.5 




.2 


3.3 


1,001 


to 


2; GOO 


Z • 1 




0.9 


2A 


A 


23.3 


501 


to 


'^^^•#000 


1 1 
1 • 3 


• 3 


2.9 


2.6 


1.3 


6.7 


301 


to-- 


500 


Q 
.9 


,.7 


/ 2 . 9 


2.0 . 


\A 




201 


to 


300 

V. 


.8 


.7 


1.5 


1.6 


2.0 


0 : 


101 


to 


200 : ," 

\ ■■ ■ ■ 


.9 


.6 


0 


3.3 


3.8 


0 




100 


.6 


.6 


0 


1.5 


' 3./ 


0 


3 


to 


50 


2.^ 


2.^ 


0 


it. 3 


16.5 


0 


-2 


to 


2 • 


77.9 ; 


8^.9 


■66.2 


'77.6 


61.1 


50.0 


-3 


to 


-50 


1.^ 


1.5 


2.9 


0 


3.0 


0 


-51 


to 


-100 


.3 


1.5 


1.5 


0 • 


2.0 


0 


-101 


to 


-200 


.6 


3.1 


1.5 


0 


2.0 


0 


-201 


to 


-300 


.6 


1.0 


1.5 


0 


.6 


0 


-301 


to 


-500 . 


1.4 


1.0 


1^ 1.5 


0 


.7 


3.3 


-501 


to 


-1,000 


3.2 


.7 


7.3 


0 


.9 


3.3 


-1,001 


to 


-2,000 


1.8 


0 


2.9 


6 


.2 


6.7 


-2,001 


to 


-5,000 


1.2 


.3 


0 


0 


.it 


0 


-5,001 


to 


-10,000 




0 


0 


0 ' 


.1 


3.3 


-10,091 


and 


less 


0 


0 


0 


0 


0 


/ 0 



Includes only cases with nonzero values. . 

, Because the central processor for Pell applications provides only a total amount for Other Nontaxable Income, we could 
not use the application value as a defaulj in our .verification of the separate contributing parts. Thus, we accepted 
undocumented values for the separate parts. To avoid the problem of failure to report aay of the separate parts during 
our documentation af forts, we set the amount reported on the application as a minimum for the total value. This tends to 
overestimate other nontaxable income somewhat and precludes negative discrepancies. . ^ 
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AMOUNTS OF DISCREPANCY FOR 
APPLICATION ITEMS BY. DEPENDENCY STATUS* 



Adjusted Gross IrKome 


15.0 


^,50^™ 


"12,098 


U.S. Taxes Paid | 


15.8 




958 


Other Nontaxable Income^ 


35^2, 




959 


Household Size 


30.5 




N/A 


Number in Pos'tsecondary Education 


18^6 . 


N/A 


N/A 


Cash/Savings/Checking 


50.3 


1,322 


1,101 


Home Value 


' 30.5 


1(»,081 


22,6(»4 


Home Debt 


23.2 


3,323 


9,339 


Spouse's/Mother's Earned Income 


17.8 


3,003 


5,m 


Social Security Benefits (Parent)3 ^ 


(»1.5 


1,581 


695 


Social Security Educational Benef its^ 


' 30. 


911 • 


137 


Dependent Student's (Spouse's) Income 


55.8 


. 1,201 


1,575 


Dependent Student's (Spouse*s) Assets ^ 


58.(». 


279 


,166 



Independent 



Ratio of Mean 
Discrepancy to 
Mean Best Value 

.37 

.78 

.90 
N/A 
s N/A 
, 1.20 

.62 

.36 

.56 
2.27 
6.65 

.76 
1.68 



Mean 
Absolute* 



Mean 
Best 



Discrepancy. 


Discrepancy^ 


Value 


22.1 


1,608 


3,79<» 


15.1 


321 


177 


22.3 - 


99<» 


639 


%A 


N/A 


N/A 


6.9 , 


N/A 


N/A 


38.9 


202 


181 


5.* 


9,15* 


2,955 


5.7 ■ 


5,7<»3 


1,725 


^ 5.8 


2,(»18 


1,154 


33.8 


1,993 


116 



Ratio of Mm 
Discrepancy to 
tiean Best Vakie 

1.81 
{1.56 
N/A 

3. 10) 
3.33^ 
2.10 



17.18 



Dependency status switchers excluded. 



^For those with discrepancy. 

^Includes only cases with nonzero values for determining percentage with discrepancy. 
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whose reported dependency status was . inconsistent with their (verified dependency 
status. For purposes of comparison, this table also shows the mean best value for all 
sample ca^es by dependency status. Independent students had appreciably higher rates 
of discrepancy than dependent students only for AGI; for most other items the rates of 
discrepancy were higher for dependent students, with the exception of the U,S, taxes 
paid and S^icial Security educational benefits for which discrepancy rates were similar 
for both groups. The lower overall discrepancy rates for independent students are 
probably attributable to their smaller incomes and f^wer assets, and may not indicate 
any substahtive differences ip" tendency to misreport application itertis.^ (It is, for 
example,^ consider^ably' easier, to correctly report ''no other income than it is to 
dpcumentthe exact amount of some other income,)' ^ ' / 

Perhaps the most revealing information in Table ^^7 is found in the column that 
shows the proportion of the mean absolute discrepancy for cases with discrepancy to 
the mean best value for all cases. The higher this proportion, the greater the amount 
of dollar error discrepancy, that typically existed on l^at item. Those items with 
larger - dollar discrepancies, however, were not necessarily those with, the largest 
number ,of discrepant cases. Independent students showed much higher proportional 
discrepancy than dependent students, probably because of their generally lower levels 
of income and assets. These higher proportional discrepancies suggest that the 
reporting errors made by independent students were generally more serious, The item * 
with the lowest proportional discrepancy for independent students and second lowest 
for dependent students was AGL The highest levels of proportipnal discrepancy were 
found for Sofcial, Security educational benefits, perhaps reflecting the difficulty 
recipients had in separating educational from noneducational benefits. ' 

For both dependent and independent students, about one-third of the cases^ had 
errors on other nontaxable income. Yet, this variable was itself composed of several 
items. Unfortunately, discrepancies for these components of other nontaxable income 
could not be determined because only the total amount was' included on the. 
application; the amounts of income attributable to each source were tallied by the 
applicant on a worksheet that was not part of the application itself. We can estimate 
tjie approximate order of discrepancies by component of other nontaxable income by 
looking at instances in which the best value for a component of other liontaxabfe 
income, exceeded the reported value for all items. According to this estimate, the five 

■ ■ ' ■ ^ e • ^ 
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^eatest contributors \o error in .other nontaxable'lncome, in order of magnitude, were 
untaxed .portion of unemployment . compensation, ^hild support, other w^elfare, 
noneduc^ational VA benefits, and thfe 1981 interes^dividend exclusion. The last of 
these will not continue to be a source of error because of changes in the definition of 
this rtem by the Internal Revenue S.ervice. 

EVALUATION OF POSSIBLE CAUSES QF ERROR 

In the previous section we reviewed the overall student error and the distribution 
of that overall error among component application items. In this section, we take a 
look at each of the student errors in an attempt to determirie the possible cause(s) of 
that error. For convenience of, reference, this discussion begins with the most 
problematic type of student error - and pr-oceeds according to the ranking of error in 
Table We first make an identification of possible causes^ followed by bivariate 

and mul^variate regression analyses. Students whose reported dependency status does 
not agree with' their best dependency status are .'treated first. Dependency status 
I switchers are excluded from other analyses in thjs chapter. * 

^.3.1 Enumeration of Possible Causes 

Using the same ground rules as described in Section 3.3.1 for institutional error, 
we identified those characteristics' of student recipients that held the most promise 
for causal relationships. Table ^-8 itemizes the variables we determined most likely 
.to* offer causal explanations. These variables fall into three groups: individual 
application items, composite application items, and environmental factors. Most of 
the possible causes of student error emerge from the application because that is the 
source for nearly^al^^of the information typically availajble for a Pell recipient. This 
implies that ihe' development of corrective actions for student error, which are based 
on the possible causes identified her6, vvill focus largely on applicatiqn-related issues. 
The last possible cause, of student error listed in Table ^-8 concerns student Selection 
fc5r validatfon. Because of its probably significant impact on student error reduction 
and its relationship to institutional compliance with validation requirements, it is 
treated as a separate topic in Chapter 5, and is touched upon only tangentially in this 
discussion. . * 

i 
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TABLE *-8 K ^ 

ChXrACTERISTICS USED TO 
ANALYZE POSSBLE CORRELATES OF STUDENT ERROR 



Application Items 

Adjusted Gross Income ' 

U.S. Taxes Paid * * 

Parents' Marital Status 

Student's Marital Status 

Father's/ Applicant's^ Earned Income 

Mother's/Spouse's Earned tncome 

Claimed by Parents, 1981 I- 

Lived with Parents, 1981 

Medical/Dental Expenses ' . 

Cash, Savings, and Checking 

Dependent Student's (and Spouse's) Income 

Dependent Student's (and Spouse's) Assets 

Tax Filing Statusr(and Tax Figures Used) ^ 

Household 'Size 

Number in Postsecondary Education ' 
Composite Application Items 
Dependency Status 

Income (AGI + AFDC + Other Nontaxable Income + Parent's 

Social Security Benefits - U.S. Taxes Paid) 
Net Worth 
Home Equity 

Environmental Factors * 



Type and Control of Institution Attended 
Problems Reported with Application\^ 
Application Processor 
Validation Status and Compliance ^' 
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Simple Bivariate Causal Analysis 

^ - - r • • ■ • • 

In this section we reviejv rates and amounts of error for the various application 
items, gpoups -of application items," and other possible causes of student error. The 
student characteristics reviewed for error were usually application or reported values, 
since these were the only values that were available, for decisions about the need to" 
validate students with particular characteristics. Error for these reported 
characteristics is summarized by rates and average amounts of error, overaward, and 
underaward for either a single characteristic or the joint relationship of two 
characteristics on error. The results of these causal analvses provide data for the 
development of corrective actions. ^ 

> * 

As we indicated above, the order in which the possible causes of error are 
treated corresponds roughly to their seriousness. Student application-item 
characteristics are covered first, in order of their dollar contribution to error, 
followed by characteristics n6t related to specific application items. The presentation 
order of the 10 causal student error topics is: 

\ 

* \ 
Dependency status 

Reported income 

Household size 

Net worth v ^ ^ 

Number in postsecondary education 
Tax filing and marital status ^ 
Dependent student income ) - 
Difficulty with application"'^-''^^ 

1 

School of enrollment ; 
Application processor. 

Analysis of Dependency Status 

The largest single source of student error was due to students whose reported 
dependency status was not the same as their verified dependency status. We call these 

• '^-^^ ' .1 \ . 
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cases dependency status switchers. Table i^-S shows. the numbers and percentages of 
cases for all the combinations of reported and verified dependency status. Overall, 5.5 
percent of the cases applied using the incorrect dependency status, with the 
overwhelming majority Of the switchers (5.1 percent of the sample, 93 percent of all 
switchers) being dependent students who applied as independent. This, of course, was 
the direction Which usually produced a lower SAI and larger award because parents- 
income and assets were usually greater than those of students. Translating these 
percentages into program-wide estimates suggests that more than 139,000 Pell 
recipients applied with an incorrect dependency status, with 129,000 applying as 
independent students when they were dependent according to Pell definitions and only 
10,000 erring in the opposite direction. , >. 



A more detailed briakdown of vthe implications of intorrectly reported 
. dependency status is shown in Table k-lO. The figures reported in the left colamn are 
dependency status error excluding students whose SEP/FAT/Invalid SAR Errprs make 
them ineligible. Some of these same figures are reported in Table student error 
by application items. The dollar figures in the right column are slightly higher because 
they consider recipients with SEP/FAT/Invalid SAR Errors as eligible. It is interesting 
to note that that those few recipients who incorrectly applied as dependent students 
had a very small mean error, and a very large proportion with zero errofl This group 
appears to be made up largely of low income students from low inco'me families, such 
that a change in dependency status had little or no affect on SAI. Even on a program- 
wide basis, this group had negligible dollar impact on error. The group which 
accounted for nearly all of the dollars awarded in error was students who applied as 
independent and were verified as dependent. Even though more than a quarter of 
these dependency status switchers had zero error (the SAI computed on the parents' 
income and assets remains the same as that computed from student income and assets, 
probably zero, because of equally low family contributions), the mean error was over 
$500. 
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Dependency status is a corgjiosite derived from the answers to three questions 
asked for both the base year (1981) and the current year (1982): lived with parents in 

1981, lived with parents in 1982, supported by parents in 19^1, supported by parents in 

1982, claimed by parents in 1981, claimed by parents in 1982. (For married^tiJkents, 
only the three items for 1982 were considered.) We are, therefore, int^ested in 
knowing which item or items among the verified items were responsible for t^e change 



TABLE it~9 

r 



REPORTED VERSUS VERIFIED DEPENDENCY STATUS 



Verified Dependency Status 



Reported 

Dependency 

Status 





Dependent 


Independent 


Dependent 


N= 1,958 

Weighted N = 2,139 
Weighted % = 5S.6 


N=12 

Weighted N = 1^ 
wcignieQ 70 = 


Independent 


N = 68 - 
Weighted N = 186 
Weighted % = 5.1 


N= 1,137 

Weighted N = 1,312 
Weighted % = 35.9 




63.7% 


36.3% 

r. 



59% 



ERIC 



4-19 



107 



TABLE *-10 



DEPENDENCY STATUS ER^ROR 



Treating Cases with SEP/FAT/In valid SAR 
Error as 



Ineligible! 



Eligible 



All Dependency Status Error 
Mean Error ($) 

Percentage of Switcfters with 
Zero Error2 (%) 

Trogram-Wide Estimate 
($ Millions) 



^65.00 
32.5 



508.7^ 
28.6 
70 



Independent to Dependent 

Mean Error ($) 

. Percentage of Switchers with. 
Zero Error2 (%) 

; Program-Wide Estimate . 
■ ' ($ Millions) 

Dependent to Independent 

Mean Error ($) 

. Percentage of Switchers with 
Zero Error2 (%) 

Program-Wide Estimate 
Millions) ' . 



500.20 
28.8 

-33 AO 
-.3 



51^7.00 



■33 AO 



^The values in this column are 
error by application item. 



in Table which presents net student award 



2a certain percenjBge of dependency status switchers hiave zero error because their 
Student Aid Index did not change. These cases are generally students of low income 
families with very low incomes themselves. Thus, whichever figures are used the SAI 
\ is usually zero. ' 
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in dependency status and .the source of the problem on the application. For students 
who applied as independent and were found to be dependent, our analysis reyealed the 
following: 

• 22.^ percent were independent according to all 1981 items, but verified as 
dependent by one or more 1982 items. ^ 

• 21.1 percent were independent according to Clairr^^d by Parents in 1982, 
but verified as dependent by both of fhe^ther 1982 items. 

^ • 16.8 percent were independent according to Claimed by Parents in 1981, 
but verified as dependent by both of the other 1981 items. 

These breakdowns, which are not mutually exclusive, suggest that students with 
independent to dependent dependency status error were most often in error on the 
1982 items, which were largely prospective when the application was filled out, and on 
the lived with parents and supported by parents items, which were difficult to answer 
and are difficult to document. 

Any effort at corrective actions to reduce dependency status error requires a 
profile of those recipients most likely to have this error. Thus, we have investigated 
the differences between reported independent s^iidents whose verified dependency 
status was dependent and those whose verified dependency status was independent. 
The results for three student chlTracteristics— age, marital status, and household size- 
that suggest differences between the two groups are shown in Table ^-11. Dependency 
status switchers, those students who reported independent status but were verified as 
dependent, were more likely to be younger, single, and have a household size of one. 



V 



A summary of case, student, and insjit!*ional error by dependency status appears 
in Table 4-12. Separate columns are presented for students whose reported depenency 
status and verified dependency status agreed and tho|se whose status changed. For 
both student and case error, mean net error per recipient with error for independent 
nonswitchers is low but positive, with slightly higher error for dependent nonswitchers. 
Students who reported as dependent but were verified as independent had negative 
mean net errors. The highest amounts and rates of error for both case and student 
errors were found for independent to dependent status switchers. Institutional error, 
not surprisingly, varied considerably across dependency status. 
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TABLE Ml 

COMPARISON BETWEEN INDEPENDENT STUDENTS WITO VERIFIED 
. ;TDEPENDENCV status of independent and DEPENDENT 
ON THRE^ STUDENT CHARACTERISTICS ^ 



Initicilly Claimed Dependency Status 



Student Characteristic 



Independent 
(Nonswitchers) 
(%) 

(Weighted N = 1,301) 



Dependent ' 
(Switchers) " 
(%) 

(Weighted N = 186) 



Age 



Less than 20 
20 to 25 
Over 25 



8.2 

'U6.7. 
1^5.1 



21^.5 
■63.5 
12.0 



Marital Status 



Married ~~~) ^ 

Separated/Divorced/Widowed 

Single 



Household Size 



26A 
68.7 



9.0 
10.5 
80.5 



One 
Two 
Three 
Four 

Five or more 



21.1 
15A 
10.3 
6.9 



70.6 
17.3 
6.3 
2.1 
3.8 



4-22 



110 



V 



TABLE 4-12 

AMOUNTS AND RATES OF CASE, STUDENT, AND 
INSTITUTIONAL ERROR BY DEPENDENCY STATUS 



Reported and Verified Dependency Status Agree 
Dependent ' Indepen dent" 

1N=1,«« (N=ri,lM) 



Mean Error 
per Recipient 
w/Error 



Cases 
w/Erroc 



Mean Error 
per Recipient 
w/Error 

(5) 



Cases 
w/Error 
(%) 



ChanRe From Reported to Verified Dependency Status 
Dependent t o independent" Independent to bependenF 
IN = 12) (N = <6™ 



Mean Error 
per Recipient 

w/Error 
($) 



Cases 
w/Error 
(%) 



Mean Error 
per Recipient 
w/Error 

m 



Cases 
w/Error 
(%) 



Case Error 
Net 

Overaward 
Underaward 



70.70 
298.14 
-267.62 



67.0 
44.2 
22.8 



41.00 
376.80 
-237.91 



46.6 
24.7 
21.9 



-93.68 
975.00 
-300.89 



66.5 
8.3 
58.2 



594.55 
845.88 
-390.00 



90.3 
72.0 
13.7 



Student Error 
Net 

Overawcu-d 
Underaward 



^il.75 
27.5.92 
-240.34 



53.0 
40.5 
12.5 



36.85 
391.18 
-229.83 



16.0 
11.9 
4.2 4 



-30.67 
1,087.00 
-363.50 



41.8 
8.3 
33.5 



551.90 
803.68 
-889.00 



75.J 

r72.0 

3.0 



Institutional Error . 



Net 

Overaward^- 
Underaward 



-11.01 
294.30 
-237*75 



26.3 
9.7 
16.6 



4.13 
342.99 
-239.86 



36.0 
( 15.5 
20.5 



-63.01 
172.00 
-186.47 



49.9 
8.3 
41.5 



42.67 
496.57 
-296.21 



42.6 
21.3 
31.3* 
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Analysis of Income 



■ One dimension of possible discrimination between cases with error* and no error 
is reported income. Income rather than AGI was used as being more representative of 
the family's full income strength. Tabte*^-13 shows student error by income ranges for 
dependent recipients. Clearly, the Higher the income' involved, the higher the 
percentage of cases in error. There wa$ not, however, much variation by income group > 
in'the mean amount of net error per recipient with error. Cases with overaward and 
underaward followed the^same pattern in terms of percentage of cases with error, but 
a more unusual pattern in amount of error, with the two extreme income groups having 
the lower values. While reported income tended to miss the mark more often for the 
highest income group, it had, on the average, less payment consequences than income 
reporting errors mTade by lower income families. 

Table ,^-1^ looks at the same factors for independent recipients. The income 
ranges, however, have been modified to be more meaningfirf* in portraying the 
economic condition of this group. In general, the distt^ution.pf ^ error cases was 
similar to that of dependent recipients; the percent of cases in error for aHj^three 
measures increased as income increased. The payment significance of that error, 
however, was different. For mean net error per recipient with error and for mean 
overaward, the mean payment error decreased as the income increased. For the 
$12,000 and over range, the mean net error was only $17.33 per case with error, even 
though slightly more than 50 percent of the cases in that range were in error. On the 
other end of the scalp, only 16 percent of the lowest ^ange had reported income error, 
but it amounted to over $780 per case. 



In general, these tables show that there is a discernable tendency for the rate of 
s^error to be correlatec^o recipient income, but that the magnitude of error did not 
vary much by inc;^^ range for dependent students. For independent students, 
however, the large errors were in the lower income groups, even though the frequency 
of error was less. , 

Our final analysis of error by income looks at student error for all of the 
combinations of reported income by verified income. The results for dependent 
students are shown in Table ^-15. An examination of these data reveal that the 
percentage of cases with error was generally higher for those stduents with higher 
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TABLE 4-13 

STUDENT ERROR FOR DEPENDENT STUDENTS BY REPORTED INCOME^ 



Reported Income ($) 

0 - 5,999 / 
6,000 - 11,999 
12,000 - 17,999 
18,000 - 23,999 
2'f,000 and over 



Ca^s 
W/Er>or ^ 
(%) 

28.6 

'fl.7 , 

62A 

72.5 

8't.8 



Mean Net 
Student Error 
Per Recipient 
■ W/Error 

($) 

1^^6.66, 

l'tl.22 • 
IW.ll 
176.12 
151.09 



Cases 
W/Overaward 
(%) 

21.6 . 
32.7 

63.7 



Mectn 
Overaward 

($) 

2^^.11 

291.00 
297.81 
■ 235.23 



Cases 
W/Underaward 
(%) 



7.1 
9.0 
16.8 
16.1 
21.1 



21 ' 

. Incpme = AGI + AFDC + Other Nontaxable Income + Parent's Social Security Benefits - U.S. Taxes Paid. 



st;udIent error for independent students by reported income^ 



Reported Income ($) 


f 

O^ses 
W/Error 


Student Error 
W/Error 


W/Overaward 


ivicaJl 

Overaward 


W/Underaward 
I*/ 


Mean 

Underaward 


-- 0 


- 2,999 . 


15.8 


780.17 


15.6 


795.66 ^ 


.2 


-355.00 


3,000 


- 5,999 




17 


20.^ 


572. « 


3.6 


-286.31 


6,000 


- 8,999 


21.7 


* 275^^ 


16.5 


^^7^.39 


5.2 


-357.3* 


9,000 


- 11,999 


'»2.2 


198.55 




3*2.29 


10.5 


-233.39 


12,000 


and over 


50.5 ' 


17.33 


27.5 


288.30 
* 


23.0 


-306.42 
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TABLE 4-15 



STUDENT ERROR FOR DEPENDENT STUDENTS 
BY DIFFERENCES IN REPORTED AND VERIFIED INCOME 



sported Income ($) 
I - 5,999 



0 - 5.999 6.000- 11<999 



25.27% 
$ 33.73 
' (n i 66) 



1*5. l%% 
$ 311.82 
(n = 16) 



Verified Income ^ 
($) ' 

12.000 - 17.999 

19.01% 
$1,525.00^7 
(n = l) ' ^ 



18.000 - 23.999 

100.00% 
$ 818.00 
(n = 1) 



.,000- 11,999 



2,000 - 17,999 



8,000 - 23,999 



:f,000y- 
terns per cell: 



16.62% 
$ -52.97 
(n = 5) 

87.76% 
$ -210.83 
(n = 7) 

100.00% 
$-1337.00 
(n=l) 



$ 90.67 
(n = 203) 

53.56% 
$ -251. §3 
(n = 7) 



57.99% 
$ " 372.01 
(n = 27) 

61.10% 

$ m.i? 

(n = 301) 

100.00% 
$ -28.52 
(n = 6) 



Cases with Error (%) 
^ Mean Net Student Error Per Recipient with Error ($) 
A blank cell indicates no cases in that cell. 



100.00% 
$ 822.03 
(n = 7) 

SI*. 25% 
$ 3t*6.7& 
(n = 16) 

70.72% 
$ 179.93 
(n = 238) 

100.00% 
$ 83.69 
(n = 2) 



Income = AGI + AFDC + Other Nontaxable Income + Parent's Social Seci^ity Benefits - U.S. Taxes Paid 



verified family income. The mean dollar, amounts of that error were also greater for 
those with higher verified income. Both the percentages and amounts if error were 
lowest for famines with less than $12,000 reported and verified income. The direction 
of student error was consistent with expectations. Students wht^se reported family 
income was greater than t+ie verified income showed substantial underawards while 
those who underreported their income sliowed overawards. The figures across the 
diagonal, which represent families with similar reported and verified income, account 
for the overwhelming majority of cases. Both the percentage of ckses with error and 
amounts of error increased with higher^ income but the dollar amounts of error were 
among the lowest. 

^ \ 
The results for independent students, shown in Table ^-16, follow a similar 
pattern, with lowest rates of error shown for those with lower verified income. 
Because of the smaller number of cases involved in the higher income levels, 
discernable patterns in amounts of error were harder to detect. In general, however, 
errors along the diagonal were lowest in both percentage of cases with error and mean 
error. 

Analysis of Household Size 

& _ '. \ 

Table *-17> reports the distribution of student/ error by reported size of the 
recipient's family. For dependent students there was no definitive pattern of error to 
suggest that any ckusal relationship between student error and dependent recipient's 
household size existed. Much the same can be said for the household size of 
independent recipients. While enror frequency was significantly less when the student 
ha^ no dependents, the magnitude of the error was little different than for other 
incTome groups. Otherwise, the frequency of cases in error was fairly evenly spread 
over income groupings. The one exception was the "6 or more" category, which had a 
substantially higher rate of incidence, the difference being in the underaward 
category.. However, the number of cases involved was relatively small and the 
difference may reflect the influence of only a few cases. 

Analysis of Net Worth ' ^ 

The distribution of student error according to ranges of net worth re^^led 
nothing startling for dependent students. Table ^-18 illustrates that, except for an 
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TABLE ^16 



Reported Inj:ome ^ ($) 
' 0 - 2,999 



3,000 - 5,999 



6,000 - 8,999 



9,000 - 11,999 



i2,ooa + 

Items per cell: 



STUDENT ERROR FOR INDEPENDENT STUDENTS 
BY DIFFERENCES IN REPORTED AND VERIFIED INOJME 



0 - 2.999 

2.05% 
$ 568.21 
(n = 8)' 

1&.36% 
$-'t39.85 
(n = 3) 



3.000 - 5.999 

$'t50.f9'- 
(n = 10) 

. 11.50% 
$ 9't.09 
(n = 3't) 



100.00% 
$-70.00 
(n = 1) 



Verified Income ^ 
($) 

6,000-8.999 - 

$ 't6.00 
(n = l) 

62.'t8% 
$ 550.28 
(n = 12) 

15.97% 
$ 171.35 
(n = 19) \ 

22A3% 
$ 2'tl.73 
(n = 2) 



T 



Cases with Error (%) 

Mean Net Student Error Per Recipient with Error ($) 
A blank cell indicates no cases in that cell. 



9,000 - 11.999 



66.69% 
$ 232.31 

59.57% 
$ 't80.35 
(n = 6) 

33.2't% 
$ 86.86 
(n = 31) 



12.000 + 

100.00% 
$1,363.00 
(n = 1) ' 

100.00% 
$l,'t38.00 
(n = 1) . 

100.00% 
$ 206.00 
(n= 1) 

92.5'f% 
$ 3$6At^ 
<n = 13) 

50.35% 
$ 98.80 
(n = 29) 



Income = AGI + AFDC + Other Nontaxable Income - U.S. Taxes Paid 



120 



ERIC 



TABLE 



STU^NT ERROR FOR DEPENDENT AND INDEPENDENT STUDENTS BY REPORTED HOUSEHOLD SIZE 



• . ■ ■ ■? ■ 




' Mean Net 










• ' - ""■■*. '.■ 




Student Error 












Cases* 


Per Recioieht 


Cases 






Mean 




W/Error 


W/Error . 


. W/Overaward 


Overaward 


W/Underaward 


Underaward 


Reported Household Size 


(%) 


($) 


(%) ' 








Dependent Students 






* . '■ .. . 










49.3 


171.^6 


W.Q 


289 20 






•' '. ■. ■ _, 3 


,60.1 


180.16; 


'f6.9 


292.08 


13.2 


-216.85 


•■•.■■■.*,.■ ■ U 

■| ■ . . - , 


." 5K0 


125.66 ». 


37.8 


269.86 


13.2 


-286.^2 


56.6 


171.18 * 




, 275.60 


12 1 ' 




6 or more 


't6.1 


110.01 


31.7 


2^8.68 


1^.3 


-196 X2 

* ' ; 


Independent Students; • ' 














■ ' 1 ■,- 


10.9 


352.89 • 


8.5 


517.^5 


2. It 


-228 84 • 


9 

** • •„■■'■ 




'i'7/i Q/i " 


• 




5 .4 


-250.94 


• . 3 V . ■ ■ . 

() ■* . ^ > * 


33.2 


. «9.^6 

y • . * ■* ■. 


29.2 


■ 508.78 


'if.O 


r 150. 16 • 




31.8 ■ 


655. 3't 




688. Oit 


1.2'- '--' ■ ■ 


; -158.57 




35.9 ^ 


398'. 86 


, ' 2^.2, 


■8Z5.^^ 


'7 ■11.1. 


-554.60 


•, 6 or more ■ ' 


V ^6.3 / 


. 396^25?^ 


' ' 3it.l 


69^.00 




" -432.92 
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TABLE *-18 

STUDENT ERROR FOR DEPENDENT STUDENTS BY REPORTED NET WORTH * 



t 

it 

Reported Net Worth ($) 


Cases 
W/Error 
(%) 


Mecin Net 
Student Error 
Per 'Recipient 
W/Error 
($) 


Cases 
W/Overaward 
.(%) 


Mean 
Overaward 
($) 


Cases 
W/Underaward 
(%) 


Mean 
Underaward 
($) 


0 - 1,000 


• 

36.5 


162.33 


\ 29.8 


2't0.29 


6.7 


-181.39 


1,00^ - 5,000 


55.1* 


203.13 


1*6.3 


267.30 


9.1 


-125.38 


5,001 - 15,000 


1*7. I*- 


153.11 


35.7 


271* . 62 


11 7 


-217.02 


15,001 - 25,000 


' 58.2 


171.78 


1*7.5 » 


267.70 


10.7 


-2W.06 


25,001 - 'to, 000 


71.1* 


150.30 


1*9.1* ^ 


321.33 


22.0 


-233.39 


*0,001 - 60,000 


69.7 


65.1*7 


1*6.2, 


258.21 


22.9 


-328.20 


60,001 and over 


56.8 


139.60 


1*0.7 


332.39 


16.1 


-3't6.60 



V - 

Net Worth = Home Equity + Real Estate/Investment Equity + Business/Farm Equity + Cash/Savir^gs/Checking 
(negative values set to zero before adding) 
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occasional variation, probably as a result of small cell size, both the frequency and 
magnitude of error were fairly evenly distributed. Error was somewhat ^ss likely to 
-occur in the lowest net worth range, but that was probably a residual of related low 
income since this level of net worth would have no impact upon the SAI calculation. 

The analysis of net worth for independent students was divided into those with no 
dependents and those with two or more, assuming that different financial 
circumstances were represented by these two groups. The top of Table ^-19 reports 
the results for independent recipients without dependents. Other than the expected 
fact that net ^rth was normally underreported rather than overreported, nothing 
revealing was seen. The number oij:ases involved in the net worth ranges over $2,000 
makes any interpretation questionable. ? 

When independent recipients with dependents were reviewed, the expected 
pattern emerged. The bottom of Table ^-19 shows the general tendency for the error 
rate to go up with increased net worth. (The ranges used for this part of the table are 



the same as those ^sed for dependent students.) The magnitude of error had no 
particular pattern, except that the $^0,001-60,000 range, had both the highest mean 
overaward and the highest mean underaward. Again, however, the number of cases 
precluded attaching any high level significance to that finding. 

Analysis of the Number in Postsecondary Education 

When the number of family members enrolled in postsecondary education was 
considered, the results revealed little of a possible causal nature. As shown in Table 
^-20 there was almost no difference in the percentage of cases with error between 
dependent students who were the only postsecondary enrollee and those who had other 
family members enrolled. There was a slightly greater tendency for the former to be 
overawarded, and the latter to be underawarded, but not enough difference to suggest 
that number in postsecondary education was a cause of error. 

Table ^-20 also displays the distribution of student error for independent 
recipients. While those cases with only one student in the family were less likely to 
have error than those with more than one, the mean net error for those with error was 
little different. 
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TABLE «-19 



STUDENT ERROR FOR INDEPENDENT STUDENTS BY REPORTED NET WORTH* 



] 



Cases 
W/Error 
(%) 



Mean Niet 
Student Error 
Per Recipient 

W/Error 



Cases 
W/Ov^award 
(%) 



Mean 
Overaward 

($) 



Cases 
W/Underaward 
(%) 



Reported Net Worth ($) 

Household Size of 1 ' 

0 - 500 20.8 627.17 18.6 750.65 2.2 

501 - 1,000 22.0 10't2.82 16.0 liti^2.75 6.0 

1,001 - 2,000 li^.Z \ 76.26 V I'^-S 76.26 0 

2,001 - 2,500 50.0 -386.00 0 ' 0 50.0 

2,501 - 5,000 '61^.3 50.11 ^ 183. 23.8 

5,001 and over 50.0 167.00 50.0 ' 167.00 0 

Household Size of 2+ ' . 

0 - 1,000 21.6 278.94 17.0 427.41 4.6 

1,001 - 5,000 30.6 272.15 22.2 425.61 8.5 

5,001 - 15,000 39.6 362.37 36.9 ' 412.79 2.8 

15,001 - 25,000 1^2.2 210.84 23.5 585.96 18.7 

25,001-40,000 33.7 471.60 33.7 ^ 471.60 0 

40,001 - 60,000 49.7 161.67 26.1 877.00 23.6 

60,001 and over 0 0 0 0 0 

,^ 1 , - . 

Net Worth = Home Equity + Reai^Estate/In vestment Equity + Business/Farm Equity + Cash/Savings/Checking 
(negative values set to zero before adding) 



Mean 
Underaward 





-418.40' 
-30.00 
0 

-386.00 
-176.35 



-268.70 
-129.07 
-308.00 
-259.87 
0 

-627.0 

0 
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TABLE *-20 

STUDENT ERROR FORpDE PENDENT AND INDEPENDENT STUDENTS 
BY NUMBERTN POSTSECONDARY EDUCATION 



Number in 
Postsecondary Education 

One 



Cases 
W/Error 
(%) 



Mean Net 
Student Error 
Per Recipient 
W/Error 

(^) 



Cases 
W/Overaward 
(%) 



Mean 
Overaward 

($) 



Cases 
W/Underaward 
(%) 



Mean 
Underaward 





Dependent Students 
Independent Students 



52.1 
21.0 



181. O^t 



\1A 



290.53 
592.08 



3.6 



-255.83 
-311.99 



I 

I*) 



More Than One 

Dependent Students 
Independent Students 



W2.2 



105.77 
'tl8.89 



37.7 
35.3 



250.09 
539.98 



16.5 
6.9 



.22'f.61 
-20*. 81 
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Analysis of Tax Filing and Marital Status 

We next looked at whether or not the recipient's family f Ued a Federal income 
tax return, table 't-2t. separates those dependent cases who filed and did not file a tax 
return, by whether the parents were married or single. Those cases where a tax: return 
was filed were more likely to be in error than those who did not file. This should not 
necessarily be interpreted as implying that filers are generally less accurate than 
nonf ilers. There are several factors that may be operating here: 

• Tax filers have more documentation available to help identify error. It is 
• very difficult to determine if nonfilers have revealed all their income. In 

the absence of contrary documentation, this study had no choice but to 
conclude that there was no error. 

• . Legitimate nonfilers have, in general, very little to misreport. Thus, their 

opportunity to err was much less. 

• Even when nonfilers err, the affiounts involved were relatively small and 
had fewlpayment (fonsequences. 

A 

Where error was identified, the magnitude of that error, whether mean net, over- 
award, or underaward, showed little variance between filers and nonfilers,' whether 
married or single. 

Table 4-22 looks^t independent cases on the same filed/not filed dimension. The 
same pattern of cases in error is displayed for dependent cases, with nonfilers showing 
less error than filers. The same explanation probably applies here. Moreover, the "did 
not file" category had a limited number of cases and should be interpreted carefully. 
It should be noted that while married filers had a higher tendency to err, the dollar 
consequence of that error was considerably les^ in this study sample. 

When the tax filers were separated, into those that used estimates to complete 
the Pell application and those that used actual figures, there was an interesting 
paradox. . There was, as shown in Table ^-23, very little improvement in- the 
percentage of cases with error when actual tax return data were reported. That may 
seem surprising until one considers that the six highest sources of student error come 
from application items that do not have an IRS 10*0 or lO^OA counterpart. Thus, 



130 

1^-35 



TABLE «-21 

STUDENT ERROR FOR DEPENDENT STUDENTS BY TAX RETURN FILING 



I 



Filed 

-^All 

Married Parents 
Single Parents ^ 

Did*Not File 
All 

Married Parents 
Single Parents ^ 



Cases 
W/Error 
(%) 



56.0 
59.2 
50.3 



32.3 
39.0 
29.9 



Mean Net 
Student Error 
Per Recipient 

W/Error 

($) 



151.59 
157. O^t 
If 0.63 



150.78 
137.82 
156. 9't 



Cases 
W/Overaward 
(%) 



38.7 



30.2 
22.8 



Mean 
Overaward 



279.91 
26't.78 



283.90 
232.97 
309.37 



Cases 
W/Underaward 
(%) 



13.5 
l't-5 
11.6 



7.5 
8.8 
7.1 



Mecin 
Underaward 



-237 A7 
-221.07 
-272.53 



-286. 'f9 
-190. l^t 
-33'f.fe7 



Includes single, separated, divorced, and widowed parents. 
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TABLE. 4-22 

STUDENT ERROR FOR INDe'pENDENT STUtDENTS BY TAX RETURN FILING 



Filed 

All ^ 

Married 
Single 

Separated i Divorced , 
and Widowed 



Cases 
W/Error 
(%) 



26.5 
31.3 
25.3 
21.5 



Mean Net 
Student Error 
Per Recipient 
W/Error 

C$) /: 



3f9.9'f 
161.01 
461.66 
460.27 



Cases 
W/Overaward 
(%) 



21^0 



21.7 
21.7 
17.9 



Mean 
Overaward 
($) 



515.11 



350.84 
598V1Z . 



581.62 



Cases 
W/Underaward 
(%) 



5.5 
9.6 
3.6 
3.6 



Mean 
. Underaward 





: -285.04 
-266^90 

-149;. 82. 



Did Not File V 
All 

Married 

Sinjgle ' ' , 

0 ', Separated, Divorcecif 
and Widorvi"!. . 



16.0 
25.0 

i6;;5 

12.0 



744.65 
1036.77 
780.52 
431:31 



15.5 
« 21.7 
16.5 



779. 1^ 
1272.93 
780.52 
513.18 



'0.5 
3.3 
0 
1.1 



-422.00 
-509.00 
0 

-355.00 
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TABLE «-23 



STUDENT ERROR FOR DEPENDENT AND IISlDBPENE>feNT STUDENTS 

: BY-. T^XmHJCJ STATUS: V 



Filed 



Estimated Figures ijfp^ 
Dependent Students 



Cases 
W/Error 
(%) 



57. k 



:i^ean| Net :. : ' ' ' ''■ ■ 

Student Error ■ ' l - • 

iPer ReCq>ient . " ;. 1 ; . , .Cases - 

W/Error - -MJOveraward 

($) :'m: 



Independent Students 29 . 



165.21 
560 -.66 



■■^■■^^25.,{);;- 



.Mean 
OveraWafd 



278.33 
' 7io.7tt 



Cases ' 
W/Underaward 
(%) 



13,6 



Mean 
Underaward 



-199.65 
-353.03 



Actual Figures Used "■ 

Depehd^|»5tudents 56. 

Inde pt] | ^fi |^tudents 25.5 

Did Not File 

' Dependent Students 3'>.9 

Independent Students 17.9 



676.^*9 



'f2.7 
19.3 



27.0 
17. if 



272.32 
'f67.98 



297.99 
701 .61^ 



13. T 
6.2 



7.9 
0.5 



-252.64 
-285.7* 



-2\H.7\ 
-211. 52 
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having actual tax return figures did little, if artything, to improve the reporting error 
on those six items most responsible for student ertori-' ; 

V.^- As in the previous ^analysis, independent students had lower incidences of cases in 
error,.. but greater.. 4^^^ of error. per recipient with error, than did dependent 

students; 'Als6vit:*shoulcl be noted that while the percent of cases with error for filers' 
using actual figures was slightly lower than those using estimated data, the mean net 
error was only about half as large. Having the actual figures seemed to improve 
independent data more than dependent data." As in Tables ^-21 and ^22, the non filers 
had" the lowest rate of error. 

To test our supposition about the effect of using actual tax return figures, we 
looked specifically at those application items that can be taken directly from th^'lRS 
10^0. As can be seen from Table ^-2^, using actual AGI and tsuces paid did improve 
the accuracy of the application data, to a statistically significant degree. Earned 
income was more difficult to obtain from the 10^0, being available directly only when 
there was one wage earner in the reporting family. We are uncertain why the 
accuracy of medical/dental expenses was slightly less when using actual figures, but 
the afffect of that item on Pell Grant payment was too insignificant to even warrant 
speculation. It should be noted that this table probably understates the effect of using 
actual 10^0 figures. Like the data themselves, whether actual or estimated numbers 
were used >yas self-reported; Jby the application filer and may have been incorrectly 
reported. Presumably, j^rror' in the "actual" column of the table for AGI and taxes paid 
represents error attril>utable to causes other than the tax return, except for a small 
number of amendedV^ttirhs or copying errors. 

""^^niaiysis of Dependent Student Income 

The next bivariate* analysis conducted was a comparison of error by income 
ranges for dependent students (and spouses, if any). Tcible ^-25 displays the finding 
that dependent student Income had very little, if any, correlation with student error. 
Other than the fact that the cases in the highest income category were likely to have 
the highest overaward and the lowest underaward, there was no discernible pattern. 
This was not surprising because of the small proportion of family income typically 
contributed by dependent students and the change in regulations to permit use of 
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TABLE th-2lt 



DIFFERENCES IN SELECTED APPLICATION ITEM ERROR RATES BY THOSE 
USING ESTIMATED AND ACTUAL TAX RETURN FIGURES 



Application Item 



Adjusted Gross Income 
U.S. Taxes Paid 

Father's/ Applicant's Earned Income 
Mother's/Spouse's Earned Income 
Medical/Dental Expenses , 



Tax Figures Used 



Estimated 
(% W/Discrepancv) 

, 33.8 
36.7 
27.0 
25.1 



Actual 



(% W/Discrepancv) Chi Square^ Probability' 



32. It 
14.3 
33. if 
21.8 
28.8 



25. if 59 
99.398 
i.8'f'f 
6.015 

y2.6'f'f 



.0001* 
.0001* 

.m5 

.1039 



r 



I 

O 



a 
b 



All Chi square tests are discrepancy (yes/no) by tax return figures (actual/estimated), with one degree of freedom. 
Those with probabilities of less than or equ&l to .05 are considered significant and marked by an asterisk. 



ERIC 



/ 



139 



138 



TABLE 

STUDENT ERROR FOR DEPENDENT STUDENTS BY DEPENDENT 
STUDENT'S (SPOUSE'S) INCOME 



4:- 
I 

4:- 



Dependent Student's 
(Spouse's) Income ($) 

0 - 1,000 

1,001 - 2,500 

2,501 - 5,000 

5,001 and over 



Cases 
W/Error 
(%) 



1*3 A 
60.9 
57.7 
50.1 



Mean Net 
Student Error 
Per Recipient 
W/Error 
($) 



175.31 

lrlit.50 

129.27 



150.26 



Cases 
W/Overaward 
(%) 



38.7 

1*1^.7 
1*2.2 
30.1* 



Mean 
Overaward 

279.1*0 
256.25 
282.38 
353.i2'f ; 



Cases 
W/Underaward 
(%) 



10. I* 
16.2 
15.5 
19.7 



Mean 
Underaward 

($) 

-213.52 
-275.22 
-2Z6.7I* 
-163.00 



*■ . 1-.". 
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expected- earnings of dependent 'Students if they where anticipated to be less than 
60 percent of those reported in the previous year. 

Analysis of Applicant-Reported Difficulty with the Application 

One hypothesis about student error has been that error is related to the. level of 
difficulty experienced in understanding the appUcation form and related instructions. 
Many complaints are heard about the complexity of the form and its error-proneness. 
Thus, we asked the independent applicants and the parents of dependent applicants if 
they had difficulty completing the application form when they were interviewed in the 
spring of 1983. Tables ^t-Ze^ developed on the basis of these responses. 

Overall, there was little difference in percentage of cases with error between those 
who reported problems and those who did notv^;' percentage columns in Table 
*-21 are very similar. There was, however, .iwnie. variation in the magnitude of the 
error, with those reporting no problem with the application having, on average, 
somewhat more correct application values. Table 1^-22 dissects that error into 
selected application items and co^^pl^es the error rates on that basis. Here some 
differences in the expected direc'^^p jvere demonstrated. However, only three of 
them-Uved with parents, 1981; ^ll^ed with parents, 1981; and home equity-were 
statistically significant. The potential effject of that finding should not be overlooked, • 
as these three items contributed heav(|^ |c» the payment error being addressed. 



Analysis of the Reaj>)enl?s Sch 

Another bivar^i^t^ptrlpajl^^^^ was student error by type and control 

of institution attended,:/:/^ is not a student characteristic per se, 

we wanted to see if in5.titutl:onaL^ be a useful proxy for the type of 

student who was susceptSa^^jji^^ findings are shown in Table ii-2i. 

Students at 2-year publiq^^^i^|dlfeV^ rate of error, followed by the 

proprietary institutions, ''li^^^^^pfi^t^s^^ i^^ it-year private or independent 

institutions. Interestingly,'.^tJ^^;di$t!|ub^ error approximates the 

pricing structures of the ins^](ttiti'6nal;;ty(ie^^^ in the table. The private 

institutions enroll the larger prbp«)ihtion? 0;f ;error dases, which might relate to the fact 
that higher income students teYld to^^^^^^ mean net error per case 

with error, however, may rel§it'^ ; tc^ ^reci^ private institutions who, ' 



Difficulty 
Completing 
Ap'plicatidh 

Had Problems 
Had No Problems 



STUDENT ERROR 



Cases 
W/Error 
(%) , 

38.8 
W.7 



7 



TABLE «-26 
ElEPORTED PROBLEMS WITH APPLICATION 



Mean Net 
Student Error 
Per Recipient 

W/Error 
($) 

229.51^ 
205.59 



Cases 
W/Overaward 
(%) 

29.0/ 
31.8 



Mean 
Overaward 
($) 

381.98 
338,57 



' Cases 
W/Underaward 
(%) 

9.8 

8.9 



TABLE k-27 

RELATIONSHIP BETWEEN PROBLEM AREAS OR ITEMS ON 
THE APPLICATION AND ITEM DISCREPANCY FOR SELECTED ITEMS 



Applicants with Discrepancy 



Not 



Item 


Reporting 
Problem 

20.6 


R^Mrting . 
Problem 


Chi 
Square^ 


Probability 


AGI 


16.5 


1.892 


.1690 


Taxes Paid 


15.7 


l't.5 


, -.158 


.6907 


Other Nontaxable Income 


33.3 


27.7 


2.65't 


. 1033 


Household Size 


31.2 


23.8 


.96't . 


.^261 , 


Number in Postsecondary 
Education 


16.0 


l't.7 


.031 


/.8599 


Lived with Parents, 198,1 


16.0 


k.3 


2k. 371 


.0001* 


Claimed by Parents, 1981 ^ 


11.9 


5.0 


5.5 


.0192* 


Home Equity 


k7.2 


31.7 


3.937 


. ).QH72* 


Dependent Student's Assets 


k(>.2 




.k5(> 


.k337 


Dependent Student's Income 


50.5 




.136 


.7122 



^reed^m^'^"^'^^ problem (yes/no) by discrepancy (yes/no), with one degree of 



Those with probabilities less than or equal to .05 are considered significant, and marked b/ 
an asTerisK* 
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TABLE *-28 



STUDENT ERROR BY TYPE AND CONTROL OF INSTITUTION 



' Institutional ' - 
Type and Control : 

Less Than 2 ^ear 
Proprietary 

2 Year Proprietary 

2 Year Public 

* Year Public 
2 Year Private 

* Year Private 



Cases 
W/Error 
(%) 

38.8 



3^.6 
26.0 . ^ 

59.6 



Mean Net 
Student Error 
Per Recipient 
W/Error 
($) 

229.85 

383':io 

228. ^^5 
,227.00 
1 3^^.77 
18'*. 76 t 




'Ca;ses 
W/Overaw^d 

31.1 

26.3 

..-.>,(' 
^Av^".20i9 

32.7 

.;32.5 



Mean - / 
Overaw0rd 



Cases / 
W/Underaward 



Mean 
Underaward 



32^*. 53 , 


•»a 7 .8 


-l(f9.TO ■ 


57^^.96 . , 


8.3 


-228.33 . 


3^*8.11 


.5.1 


-258;76 > 


358.78 




-253.01 


276.55 


12.3 


-238.77 

<• - 


332.07 


I'^.s 


-261.51 
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because of their higliec;qpst, were.at and stayed at the maximum grant even >yith "best 
value" recomputation of'award amount* Tl}ere is no readily apparent explanation for 
the higher mean overaward for 2-year proprietary schocl students. . Otherwise, there 
was not much variance among-institutional types with respect to* mean overawards and 
underawards. ' - ' 

. Analysis of Student Error by Applic?ition HrocestorV 

, ... : The final bivarUte comparison concerns the application processor. Although this 
iS; not a student characteristic, there may be differences in the types of students using 
the' various processors, so its inclusion in this chaptelr is appropriate. . . ■ ' 

.\ AppUcations for ii'. Pell, Graat are handled . in several basic ways, ^^me are 
submittefd Wectly to the Pell 'central prpcessb^^^ Federal contractor that accepts 
Pell applications fpr the- Department pf Eduqation and processes them. The result is a 
Student /Aid Report (SAr) ' which summarizes 'the application items ^'^d shows the 
computed Student Aid Index (SAI), Other financial aid'k^p^^ complete a single 
appWcation for all finah^^ by the institution they are planning to 

attend .or their state*) which is processed by one of three Multiple Data Entry (MDE) 
processors: College Scholarship Service (CSS), Aqriencan^^^^.,^^ Testing (ACT) 
Program, or the Pennsylvgpia Higher Education' Assistance Authority (PHEAA). For 
students Indicating that they wish to be considered for a.iPell Grant, the MDIE submits 
data tapes of application it^ms to the Pell processor which then computes the SAI and 
prepares the SAR. T ; 

4 A review of student error by processor revealed interesting^ and statistically 
significant results, as shown in Table ^-29. We would like to emphasize that these 
differences in rates of student error reflect differences in the applicaht constituencies 
for each; processor and do not reflect the accuracy the processing itself. The 



♦Students in California submit a special state appliGation,^ but thesfe 

Calif ornia by CSSi , • ; . . . . . ; 

**In the'I980-iil quality control report oh the. then BEOG program, we investigated-; 
and reported MDE data entry errors. While/there .were some differences among the*' 
processors, overall .lerror rates were so low that this task was not repeated for this 

year. ' ^ .y' r- - •. , ^ 



> TABLE *-29 

RATESxOF STUDENT ERROR BY APPLICATION i-ROCESSOR 



Initial Processor 
Pell 
ACT 

CSS 

PHEAA^ 



i ■ Caises 
with Error* 
(%) 



56 



Cases with 
No Error ^ 
(%) 



71 
58 

56 {; 



I.* 



Weighted 
Numt>er of 

<Ccises 



90$ 
716 
1,790 
216 



♦Chi square = 78.7't, df = 3, £ .001. 
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lowest rates of student error are shown for applicants who used, the Pell processor 
only. Students in this category were more likely to attend lower cost institutions and 
two year (or less) institutions. Their lower error rates ' rnay riflect the less 
complicated financial cirpumstances expected in this group. Applicants using ACT and' 

;.CSS had intermediate student "error rates. -Schools requesting that students use these 
services include most four year institutions. The highest error rates were associated 
•with the PHEAA . students, ^^gecause this group represented only 6 percent of 

• applicants,- a few higher ^rror cases can' greatly affect the combined outcome. 
Therefor^, we would caut^Qn; about drawing any conclusions on this small sample of 
PHE A A applicants. 

*.3.3 Multivariate Analysis. ' . 

So far we have looked at possible causes of error through bivariate ar>|>lyses. 
Multivariate analysis permits the 'joint testing of the effects of several student 
characteristics on error. We have estimated two linear multiple regression models, 
one '\yith net student ,^rror as the depen'dent variable and onevwith absolute error as the 
dependent variable. ^ ^ 

The student or student-related characteristics chosen for inclusion in each model 
as independent variables .were: 

' Effective family income • 

' ■ ' 11 

Household size 

■"-■ir-^ ' . ' •, . 

Age 

Student marital status 

Student was or was not validated by institution , . * 

Type and control of institution attended ^ • 

Whether institution validates all students. ^ ^ 

Table ^-30 shows the results of each of the two (^egressiqn models. Both^models 
share the same seven independent variables, which are shqjvrt^as row entries. The last 
row shows the R-square for. each model. R-square reveals the proportion of variance 



> ' ^ ^ -.TABLE 4-30 

• V ESTIMATED DIFFERENCES IN AVERAGE STUDENT ERROR 
^ ATTRIBVTABLE TO SEVERAL STUDENT AND 

« ^; * INSTITUTIONAL CHARACTERISTICS 

/ ^ " / * Net Absolute 

' " • ^ Student Error Student Error 

.^Ifttetcept , .37.6 . i^lA 

income ♦ * ^ . 

Effective Family Income. (Thousand $) 18.1 ''iZ.Jf 



♦ . 



* Family Size . * " * 

Household Size , -35. T -31 .6 

;^ • ^ . . # \ 

Age ♦ ? ^ 

Years of Age * " 23. S 12.3 

Student Marital Status . * 

Married ' ^ 23.2 51.7 

Separated/Divorced/ Widowed 23.2 ^7.8 

Single ■ * Base Base 

Validation 

. - - f " . ^ 

Student was not Valixiated 12.1 ."'^ -13.0 

Student was Validated Base Base * 

ft ■ ^ * 

School Type * * . ♦ 

Proprietary, Less than 2 Years 12.8 -3^.0 

Proprietary, Other 82.1 Zf8.8 

Public i-Year . -21.^ -65.1 

Public ^-Year -^^^ -20.8 -^^.5 

Private 2- Year ^ , . -25.5 -^l!5 

Private ^-Year Base " Base 

Institutional Validation 

School Regularly Validated all Students ''12.0 3,8 

School Does* not Validate on Own ' Base * Base i 

^ *^ , • • ' . » 

Proportion 'of Explained Variance (R-square) . 159 » .196 



♦ Indicates that the characteristic is a statistically significant (p .05) predictor of 
student ai^ror for the model shown. ^ ^ 
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of student error explained by the independent variables. In each model the R-square, 
which was statistically significant, was fairly strong. 

The first row of the table, the intercept, shows the average error for students 
with characteristics indicated as the base responses. There are no base responses for 
income, household size, and age because th^se are continuous variables which can 
provide a single estimate of their effect on the model. 

. The first numerical column in Table ^-30 shows the coefficients of the variables 
as related to net student error. The next column is for absolute student error. 
Asterisks (*) indicate a significant relationship between the variable and error. Thus 
income, household size, age, and school type were fignificantly related to net student 
error. Similarly, income, household size, marital status, and school type wel-e 
significantly related to absolute student error. 

The numerical entries in the table represent the difference in average error 
attributable to being in those categories rather than the base categories shown by the 
intercept. Thus, for net student error, average error for the three continuous 
variables and all base categories was about $-38, the intercept. (The base categories 
wer^^ "single," "student was validated," "private ^-year institution," and "school does 
not validate all students.") For students in public 2-year schools, the average net error 
would be $-21 lower or $-59. Using the information in the table, the change in amount 
and direction of average error can be derived for the various combinations of response 
categories. 

For the three continuous variables, we have summarized some additional findings 
from the regression analysis not included in Table ^-30. On average, both net and 
absolute student error increased just under $20 for each $1,000 increase in effective 
family income (AGI plus AFDC/Other nontaxable income, and parent's Social Security 
benefits, less U.S. taxes paid). Similarly, for each increase of one in family size, 
' student error decreased by $30 to $35. Older students and non-single students showed 
a tendency toward higher error. 
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The type- and control of institution attended were not only, significant in 
delineating institutional error (Chapter 3), but also student error. However, some of 
the relationships were different. For example, in Chapter 3 we showed that 
institutional error- in public, 2-year institutions was relatively high. However, student 
error at these schools was relatively low-in fact absolute student error was lowest in 
public, 2-year schools. This may be explained by the lower income of these students. 
Similarly the relatively high student error at private, ^t-year schools is probably 
income-related. ' 

CONCLUSIONS 

, Student error accounted for about two-thirds of the net cost of error in the Pell 
Grant program and was present in more than 62 percent of the cases. The findings 
summarized in this chapter suggest the directions that efforts to reduce student error 
might take. 

• Of the components of student error, dependency status error accounted for 
29 percent of net student error. 

• Efforts to reduce dependency status error can be directed exclusively at 
students who applied as independent. 

• Much of the error in dependency status could be reduced by checking 
whether students were claimed as exemptions by their parents. 

I • The second largest source of student error, other nontaxable income, was 
difficult to verify, especially since it was not broken down into itt major 
parts on the application. ^ 

• The third and fourth largest sources of student error were household size 
and number in postsecondary education. These itetris were prospective 
when the application was completed and thus inherently error-prone. 
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CHAPTER 5 
VALIDATION ^ ^ 

In this chapter, we review the validation process, compare it to previous years, 
report our findings concerning institutions' compliance with their responsibilities for 
validation, and report the effects that validation appeared to hav^^Tiad^upon student 
error. We then discuss some of the policy implications which these findings may have 
iri both the short and long term, as a preview to the corrective actions which we will 
propose in another volume of this report. Key findings are: 

• The vast majority of institutions collected the required Verifying documen- 
tation for their students who w>ere "flagged" for validation by ED. The 
Federal tax return was the predominant form of documentation. 

• Validated students were more likely to revise their Adjusted Gross Income 
(AGI) and Federal taxes paid initial application data than were non- 
validated students. Further, those selected for validation and making 
revisions were more likely to raise their AGI and reduce their taxes paid- 
revisions that tend to lower eligibility— than were those not selected for 
validation. 

• Among students who made changes to their application, validated students 
were much more likely to show an Increased SAI, leading to a decrease in 

. award. Corrections behavior in this direction was very infrequent among 
nbnvalidated students. 

• Most institutions believed that they were unduly burdened by the expanded 
nature of the 1982-83 validation process. The reasons cijed most often 
were delays, extra work, or confusion due to the late arrival of the 
Validation Handbook; difficulty in verifying Social Security benefits and 
Veterans Administration Benefits; and difficulty in obtaining 
documentation from students. 

5.1 THE VALIDATION PROCESS 

Before reporting the findings of the study regarding the effects of Pell Grant 
validation, it is useful to review the validation process. Because the discussion of the 
findings will focus upon the effects of validation in 1982-83 as contrasted to its 
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effects in the 1980-81 academic or program year, the validation process for 1980-81 is 
described first, followed by a discussion of the 19J52c83 requirements. 



5.1.1 The 1980-81 VaUdation Requirement 

In 1980-81, the third year of a mandatory validation activity for the Pe^ Grant 
•program, approximately 325,000 applicants, or 7.2 percent of the total applicant 
group, were designated by the central processor for validation by the institution to 
which they submitted tljeir.StM<Jfen^ Reports (SERs). Selection for validation 

was both random and on the basi^^^^^^ criteria (PEC), circumstances that 

were reasoned to be related to 'applicarnt erjror. . ■■^:>. 

Students selected for validation 'W^f^ advis^^^^ as well as by comments 

on the SER, that validation by the financial aid office at the selected institution would 
be necessary to obtain payment. Applicants were advised to contact the financial aid 
office for further information about specific validation procedures and documentation 
requirements. An asterisk besi^ie the student eligibility index (SEI) was the indication 
to the financial aid office that payment should not be made prior -to validation, 
although some exceptions were permitted. 

Selected cases were to be validated according to procedures contained in the 
validation manual entitled Basic Educational Opportunity Grant Validation Procedures 
1980-81, provided by the Office of Education to all eligible institutions. Applicants 
selected for validation were required to provide: 

• A signed copy of the applicant's (and spouse's, if any and filing separately) 
IRS 10^0 or 10^0 A Federal income tax return(s) or a statement of nonfiling 

• A copy of t>te parents' 10^0 or lO^OA return(s), if the applicant was 
► ' dependent, or a parents' statement of nonfiling 

^ • Specified documentation, as require^! by comments on the SER, for 
Medical/dental expenses 
Elementary /secondary tuition 
VA Educational benefits 
Social Security benefits 

• '^^A completed Validation Form, either provided by the Office of Education 
; . (OE Form 623) or an acceptable alternative. 
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Required Data Elements to Be YaJid^ed V 

Each selected applicant was t^> be .yaU^^ following elements contained 

on the SER: ^ ' .•^''v .y/:;,^' ^-^^^^^^ ' 

Dependency status J : ^ 
Adjusted, gross incoilne: ^v^^ " ; ' - 
Federal income tax^paid • v^' ■ ; y 
Household size V ; ' - ■ ^^"^'^^^ • - ^ » 
Number in college , ^/ ^ /^^ . , 

Other QontaxablelrjfCorW^'^ . , , 

Dependent applicant jncQ^me ■ 'j^ , ■ ' ■ 

■ ■'^■'-■^^ ^' ^ ■ 

'^Ddcum'entatio^^lo^^ medical/derttai ^ex^ VA edu- 

cational b6nefit6)4;n<^^ S<x;i^ Security 1^^^ cornment 
. appeared; on the 'SER. Specific docurhehts were required. to vkiPS^ 
elements/ ismd' were specified m;th^ as well as on -th^^^^^^ 

-Form//' . ^-IV/./'.^ ■•V*;^ ■ / V;^^.;-. ^/-^ "'^^^^^^^^ 

^./Validation- Procediu«'e ■■■'V^ ■ ■ ' '■f .^^ic:? ■/•^'' '^-^^^^^^^^ 

' . 'Upon submission; Qf t^^^^ the institiitionah f iniaoc^^ 

office,personne^ were: to tomp^ values with thei corfi? 

h^rris on the SER/. If there: were: rto ou^ discrepancies, the SElt:^^^^ 

certified for payment* exceeding specified tolerances 

noted, the appii<iant was .ta^:c^^ item(s) directly^on the $ER^^ 

sign the certification in; Section office Was to retain a photocopy of <thef^^ 

corrected SER^ and/ the- a^ send the corrected SER to the c^^tral 

processorv : When thie' reprocessed SER was returned by the applicant^ it^ \ji^as' be 
compared to the photocopy ^^^t^ that all needed changes had been pro^psejoj^; If 

so, the applicant could. b^ the SER certified for payment. I^^^lidtt^al 

corrections were deemed necessary, the corrections process NV'as rq^eated/^v 



documentation was to be retained V^' - / 



Optional* Validation 



Institutions' could optionally validate applicants not selected by the central 
processor, as well as data elements in addition to those required by the Office of 
Education. If optional validation was done, any conflicting information was to be 
resolved in the corrections process. ; ' ^ 

Limitations to the 1980-81 Validations 



Use of the official VaUdatipti -Fprrn^^^ a reasonable facsimile offered some 
assurance that most required element* would be documented in a consistent fashion. 
There were, however, some shortcomings to the 1980-81 validation procedures. 

• If an applicant claimed to be independent, no documentation of parental 
• , : ; non-involvement'was required. A notarized certification was requested on 

^ ^^U^'rtjhe Validation Fprmi;iyt.^^^^ were not allowed tq withhold payment 

if the applicant was .unabi^^^^^^^^ a parental signaiture or refused to do 

so, provided all oWer were met. Since the 

requirements for those claiming independence involved only the applicant's 
(and spouse's, if any) tax return, which* does not document whether the^ 
parent(s) did or will' claim the applicant as an. exemption, independent 
status was not really validated by this process. 

• . Several of the required data elements were not readily documentabre, even 

with the instructions provided. This was especially critical in the case of 
other nontaxable income. While the required documents could verify those 
values which were reported, there was no certainty that ail sources had 
been reported. Other items, like household size. and numBer in college, 
were likewise difficult to document. In most ca^es they were prospective 
estimates and may or may not have been consistent with hisfdrical data. 
Moreover, they were to be accepted as stated unless the institution had 
contradictory information. 

• The procedures dictated to institutions were very detailed and complex, 
and thus were prone to confusion and misunderstanding. If followed to the 
letter, they created a substantial burden upon the institutional personnel 
for what may in many instances have been very marginal corrections arid 
consequent changes in awards 



5.2.1 The 1982-83 Validation Requirement 



Pell Grant validation in 1982-83 was a significant departure from what had been 
done in prior years. The proportion of applicants selected for validation was increased 
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from less than 10 percent, chosen primarily on the basis of PECs, to more than 60 
percent, including the initial period of processing when all apparently eligible 
applicants were selected. During the coursj^-k the year; approximately 1,66 mUlion 
cases were selected for validation. Although the proportion of -cases ^'^^ be validated 

.. went up substantially, the requirements for validating each selected case.were^.eased. 

••For most applicants, the inst^itutions were required oril)«;to verify a limited number of 
application valij.es from a copy of 'the Federal tax refturn for the previous year (1981) 
for independent students or the parents of dependent students. Those who did not file 
a tax return Were required to submit a signed statement to that effect. : 

. ■ ; ' -V. . . -^.j 

Required Data Elements to be VaUdated ^ ^ ^ ^ ^ '^^^^^ ^V- ^ > ^ 

V Because only the tax return wa^xr^quired for dbcUmer^tion, in most cases the; 
only application items required to be validated were' adjusted, gross income and U.S. 
income taxes paid. Applicants whose parents reported Social Security benefits did ndt * 
agree with data obtained in a tape match witii the S6<iial Security :Administration- were 
also given a comment on their Student Aid Report (SAR) to provid^ dpcur^eiitation of 
that benefit amount. The applicant's Spcia:! S^urity benefit^, sir^e;^y were^^^h 
part of the SAI cailculation, were not a: reqi4ired validation item. -^'^^^^^^^^^^^^^ 

In addition; if there was information on the^te^ return that conflicted with 
certain items on the SAR, those items then befcarne;^ validation items. For 

1982-33, this included interest and dividend income exc^^^ untaxed portions of 
unemployment compensation, other pensions and annuities, and capital gains. While 
other data elements were not required validation items, institutions were encouraged 
to check any points of inconsistency between items on the SAR and information 
available to the institution from any other source. 

Validation Procedure 

' ' • ■ , ., • ' 

The actual validation procedure was basically the same in 1982-83 as in prior 
years, as described above. One difference was that the Validation Form or facsimile 
thereof used' previously was no longer require'3, since the required items were 
generally obtainable directly from the IRS lOW/lO'tOA. Where Social Security 
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benefits were to be validate^ schools could accep^t any form of' documentation 
completed by the Social Security Administration. 



Optional Validation 



With the decrease in required validation items, there was a dorrespohding 
increase in optional items. Institutions were encouraged to validate applicants not 
selected for validation by the central processor (once the 100 percent selection wa? 
halted), as well as additional items on; ED-selec ted applicants. The:; Pell Grant 
Validation Handbook 1982>83 went to cprtsiderable lengths (15 page^V^ta <le^^^^^ how 
to accorppllsh validation of discretionary or optional items. - 

Limitations to 1;he 1982-83 Validation 

The validation procedures .for 1982-83 were clearly more straightforward and 
less complex than in previous years for institutions which did only what was required. 
There were, however, shortcomings in the 1982-83 validation process. 

• Without the required use of the Validation Form, applicants claiming 
independence did not have to provide even a notarized certification of 
authenticity from their parent(s). The only required documentation in such 
cases was the applicant's own FederaJ tax return, which does nothing to 
verify that the parent(s) did not claim^'the applicant as an exemption, nor 
does it show anything about the accuracy ' of the other criteria for 
independent status. The removal of the mandatory $^00 per person 
minimum annual income test used in 1980-81 further diluted the institu- 
tion's ability to identify possible parental financial support that would not 
be reflected on the applicant's own 10^0/lO^OA. Thus, for all practical 
purposes, there was no required validation of dependency status. * 

• , The items identified earlier as difficult to validate, such as household size, 

number in college, and other income and benefits, were no longer required 
items. These items were checked only to the extent that institutions 
elected to do so. 

• The mandatory procedures were probably less confusing to institutions, as 
noted^ earlier, except where Social Security benefits were concerned. 
Because of a change in the treatment of the student benefits, as well as 
the phase-out of those benefits, and potential difference in what was 
in^lu<ie#in the SSA documentation and what was to be reflected on the 
SAR, there was some difficulty in understanding how the validation of 
these benefits was to be performed. 

■ 8 ' 
.■ ",•» ■ ' - 



5.2 EFFECTS OF VALIDATION ' %i ^ ' ' ' 

Validation of Pell Grant application data. cante about becaMs^/the^'^^ aid 
community, the Department of Education, and .other interested parties w?%^ d^c^rned 
about the accuracy of the data being us^fd to allocsite several billion dol^X'^annMalV* ; 
Although there is a difference ofc^opinibn as to the most appropriate way tp achieve."^ 
the desired accuracy, there is little debate that some controls ^re necessary to,issi^ ^ 
that the proper applicants are receiving benefits and that the overaUl cost'^of the Pell 
program is contained by making payments only to eligible applicants.- The basir . 
question remains: "Is validation an effective ;;tool for reducing student error?"' A 
, secondary question to be answered is: »'Was*the expanded validation effort in 1982-83 
more effective than thosip in p^st -^rs?" The following discussion, with 
accompanying data, attempts tO:address^bbth 4i these questions* We first look at the 
distribution of student error oi various types, comparing validated and norwalidated 
cases. Next, we assess the'^hanges that take place in selected application values 
between first and, most recent SARs, presumably reflecting al| corrections made as a 
result of validation. We then review the effect of validation tolerances upon error 
^ correction. * 

5.2.1 Distribution of Student Error Among Validated and Non-Validated Applicants 

Earlier in this volume, the various types of stuyient err:or were defined and 
presented as an indication of the incidence of misreporting on the Pell Grant 
application. In this subsection, we look at the distribution of that error among 
validated and nonvalidated applicants as one piece of evidence to answer the questioh 
of whether or not validation makes a difference. ' 

Table 5-1 displays net student er rot", student overaward, and student underaward 
according to whether or not the sampled case was validated by the institution. The 
data were further delineated between those selected and not selected for validation by. 
ED. Among those selected for validation and actually validated, there was no 
difference in net error between those selected by PECs and those selected randomly, 
bearing in mind that random selection during the early part of the processing cycle 
. included all applicants not selected on the basis of PECs. t That validation selection 
was somewhat discriminating can be seen in the lesser percentage of cases in error 
between the two "not selected" categories. ' 

\ . ■ -5-7 ^ : 
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TABLE 5-1 



AMOUNTS AND RATES OF STUDENT ERROR: 
VALIDATED VS. NONVALIDATED STUDENTS 



VaUdated 



• ■ ; r ■ 

Not VaUdated 



I 

00 



Net Student Error 



Percentage with Erfo^' 
Greater than $2 |< 

Mean Net Error if'er ' 

Recipient, with Error ($) 



Student Overaward 

' Percentage with Overaward 

Mean Overaward ($) 

- « ~ 

Student Underaward 

Percentage with Underaward 

Meian Underaward ($) 



PEC 
Selected 

(Weighted 
N - 199) 



1^7.1 
209.5 



34.5 

12.5 
-423.3 



Randomly 
Selected 

(Weighted 
N = 1746) 



46.8 
208.1 



36.9 
329.7 

9.9 

-245 



0 Not 
Selected 

(Weighted 
N = 679) 



36.7 
261.4 



27.5 
361.7 

9.2 
-276.0 



Selected 

(Weighted 
N = 251) 



24.6 



323.1 



18.4 
482.0 

6.2 
-145.7 



Not 
Selected 

(Weighted 
N= 751) 



33.6 



243.1 



26.6 
, 358.9 

I ■'.7.1 
-192.4 
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When dvera ward and underaward were .considered separately, the discriminating' 
ability of the PECs was more apparent, not so much in terms of cases in error, but 
clearly in terms of the size of errors identified. While the percent of cases in error 
was clearly less in the "not selected" grocps, the mean overawards were not 
appreciably different from those of cases sei|cted for vaAdation.^ . ' 

The data in Table 5-1 demonstrate that cases selected for validation are more 
error-prone, but the distinction is not as great as might be expected, perhaps due. to 
the 100 percent validation selection during the early part of the year. 

Table 5-2 looks at the issue in a slightly different fashion. When one considers 
AGI error by itself, validation of ED-selected cases did make a difference, showing a 
difference in mean error of $5.30 per case for those selected but not validated and 
$8.50 per case for those neither selected nor validated. When all types of error were 
aggregated, however, there were mixed results. Absolute case error and absolute 
student error tended to be greater for validated than for nonvalidated students. Since 
the difference was not in AGI, however, it may be presumed to be in items not 
required to be validated. * 

It is interesting to note that mean institutional error was less for validated 
students, both selected and nonselected, than for nonvalidated recipients, even though 
validation was aimed at student error. One can only speculate that institutional 
personnel are more careful about the accuracy of their own processing activities on a 
validated student because they have been singled out for special attention." 

5.2.2 Changes in Application Data during the Course of the Processing Cycle 

With all the error remaining after, presumably, most of the validation activities 
had been completed at the institution, one might be temptecl to ask: "Did validation 
do any good with respect to lowering error?" If the amount of change in adjusted gross 
income. Federal income taxes paid, and Student Aid Index during the course of the 
processing cycle was any indication, the answer had to be "yes". It should be noted 
that not all changes were initiated by validation. Students submit corrections for a 
variety of reasons, including form completion errors, keying errors at the processor, 
and changes in circumstances, among others. However, the pattern of changes 
submitted by those selected for validation was so much different than that of 
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TABLE 5-2 

MEAN AMOUNTS OF ERROR: 
VALIDATED VS. NONVALIDATED STUDENTS 



Selected for Validation 



at AGI Error * 

Meart 

Weighted N 

bsolute Case Error 
Mean 

Weighted N 

let Case Error^ 
Mean 

Weighted N 

V 

bsolute Student Error'' 
Mean 

Weighted N 

let -Student Error** 
Mean 

Weighted N 

ibsblute Institutional Error^ 
Mean 

Weighted N 

let Institutional Error^ 
Mean 

Weighted N 



Validated 

$2.00 
1S16 



$2^/8.80 
1938 



$132.00 
1938 



$152.70 
1936 



$98.00 
1936 



$118.10 
1932 



$38.60 
1932 



Not Validated 

. $7.30 

221^ . 



$2't0.20 
251 



Not Selected for validati^ 
Validated . Not Validated 



$150.00 
251 



$99.70 

21^6 



$81.20 
21^6 



$116.60 
21^6 



$68.30 
21^6 



$12.1^0 



$216. ifO 
676 



$91.00 
676 



$125.'tO 
676 



$7't.20 
676 

$121.70 
676 



$15.70 
676 



' $10.5^ 
722 ' 



$227 AO 
750 



$138.00 
750 



$109.00 
750 



$81.80 
750 



750 



$62.70 
750 



Treating SEP/FAT/SAR error 
Treating SEP/FAT/SAR error 
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cases as ineligible, 
cases as eligible. 
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nonselea|ed ^^pplicanits that impact was unmistakable. Tabid 5-3 reports the 
changis i^applicaTion data focJKGfand Federal income taxes paid from the fir^t to 
the most recent^Al^ at the tina^ of oi!^ spring data collection. Of those^applicants 
making orie or more^ corrections to their application data, a "significantly larger 
^ proportion of applicants ^i|fected for validation submitted corrections to their original 
• application data than did ^those who were not selected for validation. " As acWitional • 
evidence, there was a si^ificantly higher proportion of cases.^ amqpg selected 
applicants who revised their a6i upward or their taxes paid downward than was the 
case with npnselected applicants.. These were not the expected 'directions of 
applicant-initiated corrections, without at least the threat of validation. 

Tdble 5-3, also points up the discriminating nature pf the PEC selection criteria, 
esgeciall^ for ideqjifylng linrepprted AGt While the percentage ^f PEC-sqdeeted 
applicants submitting upward adjustments to AGI was noticeably higher than for the 
randomly" selected cases, Jhe mean and medimn increases v^ere substantially higher, 
suggesting tM^the PECs idJ^itify the hiaher error cases very well, ihe same can be 
said for 4h^e identification of the need for idownwarcf adjustments to taxes paid, 
although the difference was less 3ramatic. 

O- ^ . ^ ^ 

Table 5-3 alfc suggests ^hat' institutions were fairly effective in identifying non- 
selected applfeants for dptional v^idation.* The nonselected, but yalidated group ^ 
displayed 'considerab^ly more^ correction activity than the nonselectid, nonvalidated . 
category* < ' ^ ^^"^ 

Table 5-^ translates tHfe corrections activity to changes in SAL The pattern was 
essentially the same as with AGI^a94 taxes paid. Selected a^id vacated cases ndff only- 
showed more corrections action, but^st of tha^ correction resulted iif higher SA^, 
Non-selected case^ sHowed over 50 percent CQj:rectiorl activity, Obt most /of those' 
changes resulted in^' lower SAI values, as might be »^.^xpected in applicaat-initi|ted ^ 
corrections. 

Error Removed by Validation 

Thus far, we have s^n that validation has had the expected effects on error, 
especially for those application items that represent the focus of valTdatidh efforts" A 
summary of these findings, abstracted from tables 5-2, 5-3,' and 5-^, shows the 
following: ^ 
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CHANGE^ IN ADJUSTED GROSS INCOME AND TAXES PAID: 
VALIDATED VS. NONVALIDATED STUDENTS 



Validated 



Not Validated 



I 



Change in Adjusted Gross Income'^ 
Weighted N 

Percentage With Increase 

Mean Increase ($) 

Median Increase ($) 
Percentage With No Chjange . 
Percentage With Decrease 

Mean Decrease ($) 

Median Decrease ($) 



PEC 
Selected 

82 
49.1 
6426 
4062 
41.7 
9.2 
-6423 
-3722 



Randomly 
Selected 



, 657 
31.7. 
3062 
1264 
50.3 
18.0 

-2704 
-980 



Not 
Selected 

261 
13.5 
3869 
1625 
78.0 
8.5 
-10783 
-6252 



\ 



Selected 

26 
16.4 
2351 
2460 
66.9 
16.7 
-4307 
-2800 



Not 
Selected 



157 
4.5 
1821 
616 
92.5 
3.0 
-22790b 
-14341b 



Change in Taxes Paid 
Weighted N 

Percentage With Increase 
'Mean Increase ($) . 
. Median Increase ($) 
Percentage With No Change 
Percentage With Decrease 
Mean De<:;fea«e ($) 
Median Increase ($) 



83 
18.1 
634 
394 
42.1 
39.8 
-865 
.-597 



656 
18.6 
489 

2* 

52.4 
29.0 
-682 
-387 



254 

9.0 " 

597 

380 
81.0 
10.0 
-1060 
-*486 



27 
7.6 
425 b 
425b 
80.3 
12.1 
-4697b 
-4707b 



158 
3.1 
429b 
105b 

93.8 
3.1 

-753b 
-1119b 



^ Change from first application values to last application values. Excludes students for whom only one SAR was obtained from the 
J tentraJ processor. • _ 



A ^ Me^ns and medians based on small numbers; may be misleading.' 
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TABLE 



Overall SAI Change (%) 
Decrease in SAI (%) 
No Change in 5AI (%) 
Increase in SAI (%) 



ABSOLUTE AND NET EFFECTIVE SAI CHANGE^ 
VALIDATED VS. NONVy-IDATCD STUDENTS 

• ■ 

Validated 



PEg 
Selected 

(Weighted 
N = 53) 



60.2 




52A 



Randomly 
Selected 

(Weighted 
N = 6f5) 



61.3 
17.7 
38.7 
1^3.6 

'9' 



Not 
Selected 

(Weighted 
N = 103) 



51.6 
10.3 



Not Validated 

Not 

Selected Selected 
(Weighted (Weighted 
N = 30) N = ^H) 



23.9 
6.9 
76.1 
17.0 



55.5 
5.7 



^ Change from first application value to last application value. Excludes students for whom only one SAR was obtained from the 
central processor. ' 
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Net AGI error is 81 percent lower among students selec^5l^pR6r;^yalldatidn 
and^ validated compared to those neither selected nbr||^™a^ 

Nearly half of the PEC-selected vaUdated cases" anc|Si^^th^^^^ 
randomly selected validated cases show increases in AGH^o'riipared to less 
than .five percent of those neither selected nor i^aWated,^^ A 
increases for the first, two groups arje also much larg^^^tfian that of the 
last. Increases in AGI lead to decreased eUgibiUty an^^^d, (table.5 

:/M5out ^0 percent of the PEC-selec.ted vaUdated '^^s 
percent of the randomly selected vaUc(ated cases l^ve^krr^^^^ 
' taxes paid, compared to only 3 percent of thos^i^ " 
validated. Decreases in taxes paid lead to decrea^ 
(Table 5-^3)^ 



y On the .average,; almost one-half o^vValidated stud^q 
' . Jie^s of nonvaiidated students show^-^^, increase Ip SA 
■ ■ . >a^^^ Since, increases in SAI lead to<r'^^rease^( 

' . ^ q^^^^^ to kttribute cha¥ige in that direction, wftid 

the self-intere3^ _^'l5tudent;?y primarily to the affect. oM>S 
V sh6ul(J be- j^u that, the greater ^^ertdc^^^)^ 

stud^pts to ha^^ 




ent or 
o final 
seems 
Titer to 
ition. It 



>^<iEfelc«:te<f is fipr tfcip^ who were PECf-selected. 



:/|&fr^ those randomly 




S^Tojgether/ the^ of v||Hda^i!(fh in removing error* 

; ConsJdeting tha;f'^^^^ efforts ;of validation were unlii<el>^»^ lead to any change' in > 
• sjpvera^ of ;Nthe highest sourcfe^ e^r— notably, dependenSi?^ other npntaxable, 

. ipcomfij hdusj^hqld sizp, number in postsecondary education, arid home equity--it is no| 
; .surjprising that,^^^^^^^ ^f fectiVfen^ss o$^Validatio^^bstantial error re.malns after 

: |y^^ ec^or is discussed in the ^^^^^m^ s^tion, ^ 



5.2 J The Elfiifit diyalid^ Toleranci^ URon^firror Corre^^^ 



■ :X 'W^'^ activity discussed a^ve, muc^: of it presumiably caused 

by:;vaUdatipn select^n, bn^might ask why th^re wa^io much student , error remaining 
after vgilidatidn, at feA^'t^y the definitions used in this study; /rtiere are a number of 
potential,ex^l?i|hati|n , : ^ 



Ni^t ;aU'<;orre?jfions result in agreement with the documented values used in 
theY^ti^ .to ctltu V : ' . 



■i;l\The P^H:' program validation procedures allow specified application values 
to do^i|jJ^ from the documented ^alues by as rnuch as $300' for dependent 
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^cases and $200 for independent cases; thus, some variance whicK leads to 
payment error is, by Federal policy, ignored. 

• The validation procedures for 1982-83 required verification only for AGI, 
taxes paid, and in specified cases, parents* Social Security benefits and 
selected other items if different frpm ^information contained in the 
documentation on hand; thus, many of the error-prone items found in this 
study were not routinely validated and,discrepanci^s remained in them. 

This section focuses upon the impact of the validation tolerance levels upon error 
correction. 



The Reason for Validation Tolerances 

\ 

The validation procedures used for the ^st several years have provided a 
specif led tolerance level for certain data elements being validated. Application values 
falling within these tolerances have been defined as acceptable for validation purposes' 
and correction has not been required. The tolerance levels have varied somewhat from 
year to year, but were established as a best estimate of the amount of v^jriance that 
would have little* or no effect on the amount of the final award eligibility. Thus, the 
objective of the tolerance policy, was to ignore data discrepancies that would not likely 
nftake a difference in the amount paid to the recipient, and focus resource upon those 
cases where award amounts were affected. Several factors have made this approach 




siraj^le, if not necessary: 



• The burden upon the applicant to process inconsequential corrections 
% • The Federal cost of processing such inconsequential corrections 

• The increased length of processing time, and thus delay in payment, which 
would resiyjt from these inconsequential corrections 

• . The burden upon the schools to initiate and monitor such corrections. 

r : ' '■ , ■,■ "'I' . 

Error^Rates for Discrepancies Exceeding Tolerances ' 

'.' ^ \ 

Whfen one reviews the tables regarding overall student error, overaward, and 
underaward pf esented earlier in this report, it is not unreasonable to ask why so much 
error remains after validation. It might be concluded that the institutions are not 
doing a reasonable job initiating corrections based upon^tHe documentation provided by 
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validated students. However, if one looks at the same data, but with cases with 
within-tolerance values of AG! and taxes paid f victor ed^ out, a much different picture is' 
presented. Table 5-5 shows the percentage of ca[ses where after-validation data still 
exceed the validation tolerances, for both dependent and independent recipients. 
Remember that previous er roc tables included all types of error, while this one 
./reflects only AGI and taxes paid, the only universally required items to be validated. 
The percentage of cases still in error is noticeably improved. 

Although validation data do not show the clear-cut improvement over non- 
validated cases one might expect to find, .there clearly is a reduction in error with 
regard to AGI and taxes paid when validation tolerance Is factored into the analysis. 

Another view of the effectiveness of validation* and the impact of tolerances is 
provided, by Table 5-6. The data on AGI and taxes paid are simply the aggregation of 
the validated cases in^he previ'ous table and show that over 90 percent of the 
validated cases haci application data in agreement or" within tolerance of the best 
values derived by this study. For selected nonvalidated items, "however, the, story is a • 
different one. From 15 to percent of the cases had application values exceeding 
tolerance for other income and benefits and .parental Social Security benefits. The- 
variance of other income and benefits is not particularly surprising, in that it was 
neither a required validation item, nor an easy, one. to validate routinely with 
documentation required in 1982-83. The varianqe of parental Social Security benefits 
item is le'^s understandable. However, the central processor selection criteria did not 
request validation for all cases having parental Social Security benefits. Presumably, 
the. study procedures discovered more variance than was identified by the validation 
selection process. Additionally, Social Security benefits are not easy to allocate 
among the various Social Security programs and multiple beneficiaries, so it is possible 
that the validation process overlooked spme erroneous cases that should have been 

corrected but were not. _ _ - ^ 

». ^ • 

Payment Consequences of Using Tolerance Levels 

This discussion of tolerance effects' would not be complete without an explata- 
tion of the payment consequences of limiting correction^ to cases outside the 
tolerance levels. If the tolerance levels had been perfectly constructed, there would 
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TABLE 5-5 



PERCENTAGE OF CASES HAVING DISCREPANCES IN AGI AND TAXES PAID 
, THAT EXCEED ED-ESTABLISHED VALIDATION TOLERANCES: 
VALIDATED VS. NON VALIDATED STUDENTS 



Validated 



PEC 
Selectect/ f 



Randomly 
Selected 



Not%^^ 
Selected 



Not Validated 



Not 

Selected Selected 



Dependent Students 
, Weighted N 

' ' Percentage Who^e Documented 
AGI was Outside of $300 
Tolerance (%) 

Percentage Whose Documented 
Taxes, Paid was Outside of 
$300 Tolerance (%) 



106 
9.0 

11.3 



877 
6.9 



311 
11.6 

5.8 



54 
9.9 



178 
6.6, 

ki9 



Independent Studentl^^ c 
Weighted N 

Percentage Whose Pocumenited 
AGI was Outside of $200 
■ Tolerance (%) 

Percentage Whose Documented 
Taxes Paid was Outside of 
$200 Tolerance (%) 



15.9 
5.6 



331^ 
13.2 

7.3 



137 
13.8 



6»8 



15.6 
IQ.O 



92 
16.2 

8.3 
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Adjusted Gross Income 

Independent 

Dependent 

Total 

U.S. Taxes Paid 



TABLE 5-6 

DIFFfeRENCES BETWEEN SAR AND DOCUMENTED VALUES 
FOR SELECTED APPLICATION ITEMiS: 
CASES SELEdfeD FOR VALIDATION 



Cases with 
Differences 
Outside of 
Tolerance 
(%) 



13.5 
6.7 
8.5 



#1 

■ Cases with 
Differences 
; Within 
Tolerance 
(%) , 

21.9 
19.0 
19.8 



Cases with 

No 
Difference 

(%) 



71^.3 
71.7 



Cases with 
Documen- 
tatjpn 

376 
9U 
r 1359- - 



' Independent 
Dependent • 
Total - 

All Other Income and Benefits 

Independent 
Dependent 

Total V 
Parental Social Security Benefits i 

Dependent ^ 

Sum of Items . . 

Independent -. , 

Dependent • 

Total 



♦Excludes those whose application 



7.1 
6.5 
6.6 



31.0 
26.5 



a^t.i 

1*2.0 



7.8 
lO.i* 
'9.7 



20.5 
22.6 
22.0 



32.2^ 

29.6' 

33.3 

32.2 



85.0- 

83.1 

83.7 



65.3 
1*6.1* 
51.5 



2K0, 



25.8 



366 

967;:^ 
1331*.'* 



325 
879 
1201* 



37 



- m:3 ^ • i*p 



11*60 
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dependency Status was not in agreement with documentati^ at thejnstitufion. 



have been no payment consequences, as'th^'cases where eligibility remained constant 
would have been exempt from^orrection and the cases where eligibility was affected 
by the erroneous data would have been forced to process corrections. Unfortunately, 
the determination of SAI and scheduled payment is too complex to be supported by a 
simplistic tolerance level or even the more sophisticated zero-SAI charts provided to 
institutions to approximate the effect of data changes for cases with a zero SAI. 

Table 5-7 displays the payment consequences of both within-tolerance,and out- 
of-tol.erance data discrepancies for all items. It. is readily apparent that the tolerance 
levels did not eliminate all of the inconsequential corrections, nor^dfd^Jhey require 
correction of all the consequential ones. ^ j " ' , 

In total, 72 percent oi the cases within tolerance would have had '^i^change, had 
corrections been required, and two-thirds of those changes would have resulted in net 
payment change totaling an estimated $26.,miliion. On the other hand, 77 percent of 
the cases outside tolerance levels had changes sufficient to change SAI, and 72 percent 
of those resulted in payment changes totaling approximately $38 million. 

It is interesting to note that the majority of the out-of-tolerance cases 
experiencing no SAI change as a result of validation had a zero SAI initially and 
re^tained that index. Only a few no-change cases had positive SAIs initially. On the 
other 'hand, 27 percent of the SAI-chiinge cases had no payment change, either because 
that change was so slight as to stay within the same cell of the payment table or the 
change was not realized because of the half-cost limitation. 

5.3 INSTITUTIJ^NAL COMPLIANCE WITH VALIDATION REQUIREMENTS 

Most of the responsibility for Pell Grant validation has, since its inception, 
rested with the postsecondary institutions which the applicants attend. The Depart- 
nent of Education has assumed some responsibility by matching applicant data against 
other Federal data bases, such as Social Security and Veterans Administration, but 
resolution of discrepancies is largely left to the affected institution. Most aspects of 
vahdation of applicants attending Alternate Disbursement SysterTv(ADS) institutions 
have been traditionally handled by Department of Education personnel. For 1983-8^, 
-\DS institutions iiave been given the option of doing the validation themselves. 
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PAYMENT CGNSEQUENCES OF APPLICATION DATA DISCREPANCES:' \ 
CASES FLAGGED FOR VALIDATION BY iN-TOLEI^ANCE-AND OUT-OF-TOLERANCE DIFfArENCES 



Indepeaifcnts" Dependents t. Total* 



?tal With Documentation • * - . ^"^ 

N ^' 513 1^27 im\ 

Weighted N 578 ^ ' 15^3 ' 2121 ) 

Percentage of All Cases ^^f.l 72.1 61 5 



irrcentage With In-Tolerance Differences 35 . 3 . 31.8 ^ 32 . § 

Percentage With SAI Change 37.3 86.5 ' 72^. 

- With No Payment Change (%) ^9.6 32.0 3^.7 

- With PayWnt Change (%) \ 50.^ ^ 68.0 " 65.3 

Mean Net Payment Change ($) \ il[ 96 109 

*' Program-Wide Net Payment 6 20 ^ 26 

Change ($ Millions) . 

jrcentage With Out-Of-Tolerance 

Differences 38.^ ^6.7 ^^.5 ♦ 

Percentage With No SAI Change 55.^ 12.0 22.2 

- With Zero SA1(%) r 95.0 ' 86.0 91.^ 

- With Positive SA1(%) 5.0 l^f.O 8.7 
Percentage Wi^th SAI Change ^^.5 88.0 ;^ 77*18 

- With No Payment Change (%) 28,^ 27.^ 27.6 

- With Payment Change (%) 71.5 72.6 » 72.^ 

Mean Net Payment Change ($) 278 70 ^ ' 97 

Program-Wide Net Payment 1^ , 2^ ^ 38 
Change ($ Millions) ^ 

... , 178 _^ . ^ . . 

E^^^i those whose application dependency status was not in agreement with documentation at the institution. 



Obviously, under sqch an arrangement, much of the effectiveness of the 
validation effort was dependent upon how well institutions carried out 'these 
responsibilities. This study was designefl to address this question in a number of ways, 
including: 

• A preliminary assessment of institutional compliance in the fall of 1982- 

• An anecdotal data collection in the fall of 1982 

• A self-assessment of . institutional burden and related problems resulting 
from the L982-83 validation process, collected via the institutional inter- 
views in the spring of 1983 

• A more comprehensive analysis of compliance drawn from the applicant 
* data collected in. the spring of 1983, * - ' 

5.3 A Fail 1982 Analysis of Institutional Compliance 

r 

Because of the substantive change ir\, validation pjpcedures for. 1982-83, it was 
important to assess institutional ability to provide the required validation activities as 
early in the year as possible. Consequently, a review of these efforts was included in 
the initial institutional visits to select the student or applicant sample for the overall 
assessment effort. During the qpurse of selecting that sample. Advanced technology 
field staff reviewed the files of selected applicants to determine: 

• If the applicant had been flagged for validation 

• If complete documentation was available 

• The forms of documentation provided 

en 

• The Extent of agreement between SAR values and available documentation 

^ *■ 

• The SAI changes that would result from the use of validated data, and their 
potential payment consequences 

• The extent to which institutions were validating nonflagged recipients and* 
.non-required application items. , 

The results of this initial data collection were first reported to the Department in 
December, 1982. In February, 1983, a final document entitled Preliminary Report on 
Assessments of 1982-83 Pelf Grant Validation Procedures was submitted. This report 
was based upon the data available for a statistically representative sample of 3,^90 
Pell Grant recipients at 317 RDS institutions at the time of the fall, 1982 
^ 5-21 
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institutional visits. These data obviousiy were subject to further validation activities 
and subsequent appiicant-initiated corrections, but the preiimidary results can be 
summarised as foiiows: < 

• The vasti^maj^^^^^^ of institutions collected the required verifying documen- 
tation fORj^^ir students who were "flagged" for validation by ED. Oniy 6 
peraentM-^^^^^ iiagged recipients did not have the required documents in 
their m^oi faii, i982. 



Approxim^l^iy 76 percent of the flagged recipients satisfied the validation 
requirenierii^ by providing a signed copy of their Federal tax return; 2 
percent submitted an acceptable alternative to a tax return; 15 percent 
signed a statement asserting that no tax return was, or would be, filed; and 
1 percent had no verifying documentation because they were exempt from 
validation. 

■» 

The great majority of institutions appeared to be identifying incorrect 
application entries in the cases flagged for validation and getting them 
corrected. AGI was correct in 89 percent of the cases and U.S. taxes paid 
was correct in 85 percent of the cases when values from verifying 
documents were compared with values on the application, as indicated on 
the flagged recipient's most current Student Aid Report (SAR). 

For most of the documented cases, the application item discrepancies were 
small. Only 7 percent of the documented flagged cases had item 
discrepancies that when taken individually or summed exceeded the ED- 
established tolerances. AGI exceeded tolerance in 3 percent of the cases; 
U.S. taxes paid also exceeded tolerance 3 percent of the time. 

Approximately two percent of the documented flagged cases had out-of- 
tolerance differences which would lead to a change in the student's 
expected award. For the 1.7 million recipients represented by the sample, 
these differences would translate into an estimated net overpayment of 
$3.^ million. This dollar figure can be viewed as an estimate of the level 
of institutional aoncompliance as of fall, 1982. 

Approximately 6 percent of the docurrtented flagged cases had differences 
within tolerance which would lead to a payment change. These differences 
would translate into an estimated^net overpayment of $1.6 million as of 
fall, 1982. This dollar figure can be viewed as an estimate of the dollar 
savings to the program not captured as a result of the validation tolerances 
in place at the time. - ^ 

Public institutions appeared to be most diligent in complying with the 
validation regulations, followed in order private and proprietary institu- 
^ tions. Roughly 5 percent of the fil^;|of flagged students at public 
''^ institutions werip incomplete, while 8 percent of the private institution 
files and 19 percent of the proprietary institution files were incomplete. 
The difference in proprietary school compliance may, however, reflect the 
^ " more frequent, and thus more recent, start dates of their recipients rather 
than any greater lack of diligence. 
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Many institutions were voluntarily taking step# to ImJtoYe the quality of 
application data. Institutions collected a Federal Sx^ re^ from 28 
percent of their unf lagged recipients, an alternative t<«i^j(i^ return from 1 
percent, and a statement saying no tax return wafs, or would be, filed from 
10 percent. A file comparison between the validating document and the 
student's SAR showed that for these unflagged cases, AGI was accurate 89 
percent of the time and U.S. taxes paid was correct 91 percent of the time. 

Institutions rarely verified application items that Ikre optional for 
validation, even for flagged SARs. For example, the asset items were 
documented in less than 0.2 percent of the flagged cases.' vS| ' * « 

Early indications were that institutions in 1982-^3 ^ere less likely to be 
out of compliance with the validation * requirements than institutions in 
1980-81. Approximately 11 percent of the flagged regipien^ files reviewed 
during the 1980-81 quality control study had no verifying docy mentation of 
AGI. In roughly 10 percent of the files, U.S. t^es paid w^ hot 
documented. In 1982-^3, only ^ pertent had no do<^menfation of iTCI and 
only 6 percent were missing documentation of U.^S. taxes p^d. ' 



The preliminary report provided a number of statistical tables to support' the 
above conclusiohs. Those figures are not being reported here because they have been 
superseded by moje CO mplete^data. % ^ ^ 

5.3.2 Fall, 1982 Anecdotal Data CoUection m ^ ' 

During the field worh^ many financial ai<f* administrators (FAAs) offered their 
complaints and suggestions to thp dat| collectors who visited their institutions. The 
following comments %teni^ from |wo sources. The first are concerns voiced by FAAs 
during the sampling and Compiling of data in the field. The other source is a summary 
of the comments provided by the data collector^during a debriefing held on December - 
1 1, 1982. The comments have been organized by the following five topics: 

Administrative problems 

Communication problems with ED regarding all issues, including valjef^tion' 

• — •^ii 
Probleips with the Pell Grant applicatioQ process " ;f 

Problems with validation procedures ^ 

Recommendations to improve validation procedures 
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Overview 



The majority of FA As who commented from the sampled institu 
acknowledged the gositivd effects of the' expanded validation effort; ho^vever. thej- 
believed that the many problems ahd delays caused by costly, inconsistent, and - 
unrefined validation procedures cannot be overlooked. A v^^t^pf factors, ranging V 
.frorn overburdened financial aid office staffs and inadequate^feinication with ED, 




itributed to the 



targeted at V. 



act, many 
iyitable and 
lation is a 



to difficulty in oBtaining documentation from government age 
dissatisfaction of many FAAs. These complaints and suggesti 
program inefficiency and not at the merits of the validation ef f 
FAAs considered the problems associated with insi:e^sed valid 
actively sought to resolve them.;>-One FAA's remark that "the 

burden, however, it is acceptably Jf .it does provide more*^pt»^^^o^?Se%ediest* » 
students" summed up the attitude, of rtiany. It is important to''noWH'fiaf%A?^^ 
previously had validated percent of their stud^i;its'° of their bwX 
significantly fewer complaints and reacted positively to the'expajisiofi oi'%li^ation. . 

During the debriefing, the data cp\\iKtovsy^r\ioUi^:knA /expoundedk^;,i^'pn "the-'-i^^ 
opinions expressed by FAAs^;. concerning expaind'ed ' ' 'som'^roi#"\(fe -'xlkta • 



collectors reported that small schools could han 



ijli't vali'Satibri but that 




iver^.^od tbat Vi^r.d^D^nds very much 
eff iciencyf on ho>y they vi 




large schools were overwhelmed. Others 
on the personality .of the finai^cial aid^" 

their job." Many institutions were- alrea^fS|i||g..vaU^bn similar'; to that, required by 
the Pell Grant program for all their' aj^ recipie^n^^^ ' . a> 

(especially at private institutions) or lif respons^ to the r^^ei^ 
programs. >^any proprietary schools, aware that they" ar^ suspected of was^ or fraud; 
because of a 'few well-known cases, were especially scrgpulous. bnly one college was 
reported to disagree fundamentally with validation. As one data coilecfor Observed, ''I''^ 
don't think their argument is with validation itself. I pinned them ddwn on it; it'^^ thM, 
process. They all, I think, see the need of validation." ' 

Generally, FAA3 stressed tHe need for more money to alleviate administrative 
burdens, for clear and explicit guidelines to eliminate inconsistent regulations and 
forms, and for timeliness of ED changes and announcements. As one FAA asserted: 
"Application, and validation procedures result in declining student participation," 
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fundermining the goals of the Pell Grant progrgim. B^^s^ problems 'of validation 
ultimately harm student applicants, FAAs saw^the need f pj^im'r*iiediate imprt-OvementsJv ' 

. The specific problem areas cited by, FAA^s are li/ted below. For eachrof the f ive , 
if)roblem areas, we summarize the major findings. , . 



V Administrative Pro^Jlems ^/ V ^» * 

• ' ; Despite the general acceptance of' e){pandefd ^Udat^pn, many^AAs foupjcl 

/costs; paperwork, and administrative burdens e " 

• PAAs -agreed* with .the need .to. ..validate, but" fac^ (jifficufty ih funding \ 
> : P<5sitions to: ta^^^ 

• ^ FAAs were uncertain if much money had been sav^ .{^/val^tin^^ - 
. of .the ex{>enSe of reprocessing and the cost to in^titut^c^. ^ % . m - ^K^. 



Cohnmunicatibrt, Problems With ED regarding AH Issues, Irtc^udin^^iat ^ "^'^ 



there, was a general expression of the heed ^foi^ figtear ar/& timely ' 
communication from ^D. Aj^ specific example of /this' was the jfeed for 
earlier delivery of the VaUdatlon Handbook , which slifeald.s^^ta,iJi simple 
language, the institution's responsibilities. . - ^ ■ ^_ . ^ 

Mid-year changes in regulations or payment schedules cause^)nfusioi¥,and . 
unnecessary burden. ; - ; ^ 

Retroactive changes by ED must be avoided. ' 

Institutions already validating 100 percent seemed to ad j^sf rpore easily to 
> ED'S changes. The situation was different and more serious fbr other 
institutions. , . ^ ' ^ .4 ;^^ 

There was a need for continued training and workshops, esf>eciaily ,aB tax 
^ forms. Training and communication from ED wereiconsidere'd ligss tMable 
than private sources, such as NASFAA ' 7- 



: f believed that there was a lack of support for their institutiqhs by ED 
'- ^ - as a result of poor comrpunication. FAA credibility was damaged /b)^ 
conflicting policy or regulatory interpretations by ED staff: a student who 
was told one thing by his FAA could get a different answerjby calling :ED. 
This was particularly the case with payment options and validation^ 
Another example was the perception that ED did not follow through on 
cases turned over to it. ^ 

. ^. # • 
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J^^b^l^^ the Pell Grant Application Process 
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• rieed for more explicit direction/ for the applicants, more 
;'#Qpmprehensive instructions for FA As, and improvement of the edit system. 

W ^ t^e comments section of the SAR was another area where instructions 
.\H^ <i:ould have used improvement, since students did not always read it. 

^ v • There were many problems with defining and determining dependency 
I / status, particularly for^tudents who had somewhat unusual circumstances. 
I ; Several inequities of the current definition were mentioned. 

FA As asked that reporting requirements for other income be clearer and 
; more specific. 

• The confusion over completing the application caused some students to 
hesitate filling out the form and fear the consequences of a mistake. 
There was also a concern about confidentiality. 

• FAAs considered the SAR reprocessing delays to be one of the largest 
problems they faced. They saw a need for refining the edit system as well, 
and would like the toll-free number for questions reinstated. 

Problems with Validation Procedures 

• From the viewpaint of many FAAs, validatioh should be a means to correct 
and prevent gross discrepancies on high priority items, and not to be used 
as a control or policing mechanism for relatively minor ones. 

• Government agencies, namely the Social Security Administration, the 
Internal Revenue Service, (IRS), and the Veterans Administration (VA), were 
targets of blame for delays in the validation process. Local social service 
agencies seemed to be particularly notorious for long delays in 
documenting welfare benefits received by students. 

• Regulation, changes 'regarding Social Security and VA benefits contributed 
to increased workload and delays in disbursing Pell Grants. Retroactive 
changes were the chief source of complaints. 

• Along with delays caused by regulation changes, FAAs had problems 
obtaining and using documentation, particularly tax returns. Many FAAs 
sil^gested that ED. require the 10^0, lO^OA, or an alternative as part of the 
Application for Federal Student Aid. pthers suggested that insufficient 
attention was being given to documenting those who' did not file tax 
returns. \ . % 




• • There was also the problem of documenting a student's past Sj^ncial aid 
and obtaining a financial aid transcript. FAAs found themselves in a 
peculiar situation, since they had no way of knowing if a student had 
attended another school and received financial aid. ' Many pointed out that 
. a student in default at one school can still get a Pell Grant at another. 

• FAAs commented that many families, especially low-income families, do 
not keep documents and cannot complete validation. This was of particular 
concern to fAAs since- they wanjed to assist in getting moneyvto the 
neediest students. / , 

• Some FAAs believed that the present emphasis on documentation seems to 
contribute to abuses of the Pell Grant program, with cheating becoming 
institutionalized. , ■ 

Recommendations to Improve Validation Procedures 

• There was general agreement on the usefulness of simply requiring every 
student to submit a copy of his tax return (and his parents', if dependent) 
with the SAR. *^ 

" ■ .r ■ ' . ' 

• A second discussion focused around incentives for institutions. Under 
current regulations, an institution that discovers overSwards saves the 
Federal government money, but does not help the institution or its honest 

accurate, applicants. Allowing the institution to keep some of the 
savings foF administrative costs was suggested as an incentive for doing 
100 percent validation and . doing it well. . 

• A third suggestion was to combine extensive validation for a smaller 
number of error-prone students with validation of cJnly a smafl number of 
Items—no more than four—for everyone else. 'y l^jf* | 

5.3.3 Self -Assessment of Institutional Burden and Other Problems W/ 

The anecdotal data in the preceeding subsection were collected duflng or, shortly* 
after the intensive effort on the part of institutions to comply with the 1982-83 
validation requirements. Because of the ad hoc nature of the fall, 1982 institutional 
visits, there was no opportunity to develop quantifiable data on institutional burden. 
However, it is safe to conclude that a majority of Ihstitutionaf financial aid personnel 
felt that they were unduly burdened by the expanded nature of the validation process, 
at least at the time.of our fall visits. 



In order to quantify the institutional perceptions of the burden of validation/ the 
Institutional Questionnaire (IQ) administered in the sj^ring of 1983 asked a number of 
questions about the impact of the validation etfort 1^on the institution. As can be 

" , " , ' ' ' «- * f 
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seen in Table 5-8, only 18 percent of> the IQ respondents considered i.^hp^^^^ 
validation process to .be "no problem." The 72 percent who did consider it to be a 
problem reported a number of different reasons. The most frequent (26 fjerde^^^^ 
respondents) problem^eported was the delay, extra work, or confusion b^^ght^ 
by the late arrival of the final instructions via the Validation Handbook . Difficulty 
experienced in getting documentation for Socal Security benefits caused ^^rpblem for 
/f9 percent of the respondents, followed by difficulty getting documentation from 
students (17, percent), verifying VA benefits (11 percent), and delays or difficulties 
associated with recalculating awards due to mid-year changes in treating VA benefits 
or the revision to the payment schedule (9 percent). Seven percent of the respondents 
reported difficulty understanding the validation instructions and procedures once they 
were issued. Delays or difficulties getting tax returns from the Internal Revenue 
Service affected six percent of the sampled institutions. A variety of other problems 
were reported by fiye percent or l4ss of the respondents. ' 

A comparison of the anecdotal comments and the problems reported in Table 5-8 
suggests that th> passage of time between the first and second visit moderated the 
perceived impact of the expanded validation requirement, as migh^^expected. When 
the questions were more specific, however, such as the occurrence of unusual delays in 
making Pell. Grant payments (Table 5-9), or the creation of an unusual burden on the 
financial aid staff (Table 5-10), a higher response rate wa^ received eyen in the spring 
when the worst was presumably over. ^ - 

Furthermore, the response to the questions of dfelay and burden was not evenly 
distributed across all institutions. Table 5-9 displays the ^fact that private ^-year 
institutions felt the most delays, in percentage of institutions Effected, percentage of 
students :^iec ted, and mean nuxi^ber of weeks delayed.. Public ^-year institutions were 
.the nexf most affected, followeVby private 2-year, public 2-year, and proprietary 
institutions respectively. To thj^^ extent that one can generalize the' financial aid 
processing-calendar based upon type and control of institution, the;, more in advance of 
the start of the enrollment period the work of the financial aid office takes place, the 
more frequent and more lengthy the delays reported. 

Table 5-10 shows that the distribution of burden also varied according to type 
and control of institution, with the greatest burden being bol-ne by puWic ^-year 
institutions, followed by private /f-year ,s;<|idols, 2-year colleges (both public and. 



TABLE 5-8 



PROBLEMS EXPERIENCED BY SAMPLED INSTITUTIONS 
AS A RESULT OF VALIDATION 



Response 



No Problems 
Problems 

Delays, extra work, or confusion due to late 
arrival of Validation Handbook 

General difficulty in verifying Social Security 
benefits 

General difficulty obtaining documen'lation 
from students , 

General difficulty verifying Veterans Adminis- 
tration benefits 

Delays or difficulties associated with recal- 
culating awards due to VA regulation change 
and/or payment schedule change 

, Instructions unclear /uncertainty about which items 
to validate/uncertainty about procedures overall 

Delays difficulties obtaining copies of 
tax returns from the IRS 

High volume of validation cases placed undue^ 
burden on financial aid office 

General difficulty verifying othpr nontaxable ' 
income (all other income and benefits) 

Difficulty verifying interest/dividend exclusion 

Difficulty obtaining signatures on tax returns 

Difficulty dealing with the Social Security Adminis- 
5 tration in obtaining dociumentation ^ ' 

> Delays in receiving corrections from processor 

Difficulty explaining procedures/changes 
in procedures to families 

Tolerance limits were inappropriate 

Getting students to. report taxes paid rather 
than taxes withheld . 



Percentage of Financial Aid 
Administrators With Reponse 
(N=311) 

18% : ' 

72% 



26% 
19%' 
17% 
11% 

9% 

7% 

6%" 

5% 

5% 
^% 
^% 

3% 
3% 

3% 
2% 

•2% 
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TABLE ^16 



Reported Inj:ome ^ ($) 
' 0 - 2,999 



3,000 - 5,999 



6,000 - 8,999 



9,000 - 11,999 



i2,ooa + 

Items per cell: 



STUDENT ERROR FOR INDEPENDENT STUDENTS 
BY DIFFERENCES IN REPORTED AND VERIFIED INOJME 



0 - 2.999 

2.05% 
$ 568.21 
(n = 8)' 

1&.36% 
$-'t39.85 
(n = 3) 



3.000 - 5.999 

$'t50.f9'- 
(n = 10) 

. 11.50% 
$ 9't.09 
(n = 3't) 



100.00% 
$-70.00 
(n = 1) 



Verified Income ^ 
($) 

6,000-8.999 - 

$ 't6.00 
(n = l) 

62.'t8% 
$ 550.28 
(n = 12) 

15.97% 
$ 171.35 
(n = 19) \ 

22A3% 
$ 2'tl.73 
(n = 2) 



T 



Cases with Error (%) 

Mean Net Student Error Per Recipient with Error ($) 
A blank cell indicates no cases in that cell. 



9,000 - 11.999 



66.69% 
$ 232.31 

59.57% 
$ 't80.35 
(n = 6) 

33.2't% 
$ 86.86 
(n = 31) 



12.000 + 

100.00% 
$1,363.00 
(n = 1) ' 

100.00% 
$l,'t38.00 
(n = 1) . 

100.00% 
$ 206.00 
(n= 1) 

92.5'f% 
$ 3$6At^ 
<n = 13) 

50.35% 
$ 98.80 
(n = 29) 



Income = AGI + AFDC + Other Nontaxable Income - U.S. Taxes Paid 
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TABLE 5-10 



JJREQUENCY WITH WHICH INSTlTUnONS REPORTED THAT 
VALIDAHON CAUSfiD AN UNUSUAL BURDEN ON THEIR STAFFS 
BY TYP^ AND CpNTROL OF INSTITUTION 



i 

Percentage of Institutions in ' o 
Sample Reporting That Validation 
Caused an Unusual burden on - 
Their Staffs 



Public 



it year 
(N=105) 



86;7 



2 Year 
(SJ=87) 



58.6 



Private 



H Year 
(N=75) 



82.7 



2 Year 
(N=10)r 



60.0 



Proprietary 



Less 
. Than 
2 Year 2 Year 
(N=13y (N=20) 



46.2 



For Those Reporting Burden 

Percentage Reporting That 
Additional Staff Had to Be 
""Hired Due to Validation 

Percentage Reporting Increases 
in Staff Overtime due to 
Validation 

Percentage Reporting That 
Other Staff Functions Had 
to be Dropped Due to 
Validation 



t 5.6 



17.8 



72.2 



2.0 



75.5 



4.8 



17.7 



66.1 



0.0 



20.0 



60.0 



0.0 12.5 

7 ' i 




60.0 



40.0 



12.5 



50. p 



Percentage Reporting That 
Validation Created Other 
TJ'ypes of Hardships on Their 
, Staffs 



4.4 



4.1 



11.3 20.0 

^ ft 



0.0 



25.0 
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private), and proprietary institutions in that order. In this instance, type and control 
•may be functioning as a proxy for average number of Peii applicants per institution, 
rather than as a characteristic of its own. All institutions except 2-year proprietary 
schools used the deferral of other staff functions as the primary means of handling the 
staff burden, with overtime being a distant second alternative. There was probably a 
processing calt^dar relationship to the burden responses as well, y/ith the ^-year 
institutions vi>^^|^^ would have h$d the most backtracking to do when the validation 
procedurdfe " w^/e djjgeminated)' having the ^reates^ perceived burden and the 
proprietary schools (who would have the least 1982-83 processing completed by the 
time the validation procedures were finalized) having the least amount of additional 
burdenT. 

One important facet of the reported burden and delay associated with the 
1982-83 validation process should receive special note. Tbe timing of the dissemina- 
tion of the Validation Handbook appears to have been a major factor in the problems 
experienced by institutions, coming as it did after many schoQls had presumably 
completed processing their: fall awards. The additional burden and delay would clearly 
have been less if the validation procedures had been known at the time SARs began' to 
arrive at "traditional" institutions in the early spring. While the expanded number of 
validations obviously would have affectedynany institutions, even with an earlier start, 
at least some of the burden would have been dissipated by the lesser number of data 
elements to bet validated, with a minimal net increase in burden as a result.. 
Unfortunately, thfer'e was no satisfactory way to separate the timing factor from the 
expanded number qi validations in our study data, so one can only speculate as to the 
Qxact cause. 

' \ 

. \ ■ - - » 

5.3.4 Spring 1983 Analysis of Institutional Compliance 

* • As noted in section 5.3.1, a very high percentage (94 percent) of recipients 
flagged for validation appeared to have the required documentation in their files as of 
the fall institutional visits. When this indicator of compliance was assessed during the 
spring institutional visits, the results were somewhat less indicative ©thoroughness on 
the part of institutions. The first column of Table 5-11 shows the percent of ^^s 
selected for validation which had complete required documentation in the file at the 
time .of spring data collection. Some %3 ^perqent of the cases had documentation 
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TABLE 5-11 ' 

INCIDENbE OF FILE DOCUMENTATION FOR 

STUDENTS SELECTED AND NOT SELECTED FOR^VaLIDATION ■ . 

Selected 3 Not Selected 

V for Validation For Validatiort^ 

(Weighted N=2110) ^ (Weighted N=137* 

Percentage With Complete Documentation 82.9 '66,9 



With Tax Return 66,4 



./ 



With State Tax Return .2 
With W-2 Forms ^ .2 

With Statement From Student/Parent ' - .3 



<f9.9 



With IF^S Transcript 1.5 -J ..\ 1^0 



.1 

It 

.2 

A 



With Statement of Nonfiling 12.7 [t^ 

With Other Documentation 1.6 1.1 - 

(includes use of earned income 
portion) 

Percentage With Incomplete Documentation 17.0 33.0 

1 ' •' ■ ■ ■ ■ - ■ 

Percentage Exempt From Validation .1 .1 

*^ Excludes students for whom documentated dependency status does not agree with application dependency status. 
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necessary for validation purposes, whil^ 
.1 percent were exempt from validation ia 
return', was the predominant form of docSumeh 
validation requirements in place. 

In addition tbithe assessment of complet^Validation documentation reviewed 
above, the Institutipnal Questionnaire request^^gftSormatioh .about practices followed 
to enhance the quality of student data. Table shows the institutionally reported 
practices other than required validation which presumably improved the accuracy of 
the data used to determine Pell Grant awards. Routine, consistency checks for data 
elements were predominant, but only about one-third of institutions conducted 
100 percent validation. 

When these additional' steps to increase the accuracy of student data \^ere taken, 
student error did improve. Table 5-13 shows the percentage of cases and mean error 
for net student error, student overaward, and student underaward. • While the 
percentage of cases with error was not significantly less for students attending 
institutions that indicated ' they routinely checked the consistency of Pell Grant 
application data against other file data, there was a sizeable lower mean error, as well 
as less mean overaward and m^an underaward among students at those institutions. 
<:onsidering the fact that this was a self-reported practice and that the activity was a 
Pell program requirement, schools, may havp been reluctant to admit they were not 
adhering to that requirement; the r6al difference between schools that did check 
consistency and those that did not may have been even greater. 

Although fewer schools conducted 100 percent validation (validated unf lagged as 
well as flagged cases) than did consistency checks, the pattern of results was the 
same. Table 5-1^ indicates fewer cases with error remaining and' between $33 and $68 
lower mean error when 100 perpent validation was conducted. Once again, there \yas 
evidence that institutional validation did make a difference in unresolved student 
error. 

V 

5A EFFECT OF VALIDATION UPON PELL GRANT APPLICANTS 

The validation process not only affects institutions <;;through the additional time 
and effort of receiving and reviewing documentation, but it also presumably makes 
demands of the applicants and their parents. We were interested in determining just 
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TABLE 5-12 ' 

PERCENTAGE OF PELL RECIPIENTS ATTENDING 
• INSTITUTIONS REPORTING VARIOUS QUALITY CONTROL PRACTICES 



Institutions Routinely Check 

the Consistency of Pell Application 

Data against Other File Data 

Yes 



No 



Percentage of 
Recipients 
Attending 

^ (%) 



89.^ 
10.6 



, Percentage of 
bistitutions 

(%) 



88.1 

-? 

11.9 



Instituticms Conduct 100 
Percent Validation 

Yes 



No 



32.3 



67.7 



3'f.l 
65.9 



L 
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. TABLE 5-l'3 

AMOUNTS AND RATES OF STUDENT ERROR: 
CONSISTENCY CHECKS VS. NO CONSISTENCY CHECKS 



I 



Total Net Student Error 

Percen;tage with error greater 
than $2 ^ , • 

Mean net error per recipient with 
error ($) 



Student Overaward 

Percentage with overaward 
Mean overaward ($) 



Students Attending Institutions 
That Routinely Check Consistency 
Of Pell Application Information 
Against Other File Efata 

(Weighted N=3022) 



38.7 
160.7 

29.6 
282.5 



Students Attending Institutions 
That Do Not Check Consistency 
Of Pell Application Information 
Against Other File Data 

(Weighted N=359) ~ 



^1.9 



198.3 



30.8 
367.6 



Student Underaward 

Percentage with underaward 
Mean underaward ($) > 



9.1 

-236 



11.1 
271.9 
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TABLE 5-1* 



AMOUNTS AND RATES OF STUDENT ERROR: 
100% VALIDATION VS. fiO 106% VALIDATION 



Total Net Student Error 

Percentage with error greater 
than $2 

Mean net error per rec<pient with 
error ($') 



Studeats Attending Institutions 
That Conduct 100% Validation 

(Weighted N=1175) 



36.3 



191.80 



Stu<!i-nts Attending Institutions 
That Do Not Conduct 100% VaiUdation 

(Weighted N=2'f58) 



228.10 



1 



Student Overaward 

Percentage with overaward 
Mean overaward ($) 



27.0 
3^8.80 



33.8 
361.80 



Student Underaward' 

Per<;entage y/ith underaward 
Mean underaward ($) 



7 



9.3 
-20if.00 



9.0 
-272.30 . 



J 
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how onerous the task of supplying validation documentation was perceived to be by 
those asked to comply. If the feedback obtaiijed from the Student and Parent 
Questionnaires are were true indication, the "burden" did not seem large enough for 
any great concern. '' 

Tables 5-15 and 5-16 display the frequency with which students and their parents 
remembered being asked to provide validation documentation, and any difficult they 
had understanding and complying with the instructions. The small percentage of 
students or parents recalling the request, when some 60 percent of all recipients were 
selected, suggests that it was not a great or important occurrence. No more than 10 
percffent of any category reported any problem understanding what they were to do to 
comply. Being able to comply without difficulty was a bit more troublesome, but only 
independent students and their parents had difficulty in more than 11 percent of the 
cases. It is understandable that the parents would have had somewhat more difficulty 
complying with the documentation request. They often had no involvement in 
completing the application in the first place and i< so, did, not feel any obligation to 
participate. 

It seems reasonable to conclude from these data that the requests for validation 
documentation were making no great impression upon either applicants or their 
parents, and, with the exception of the parents of claimed independent applicants, 
were not approaching a burdensome level. 

5.5 CONCLUSIONS 

For 1982-83, the Department of Education required validation of about ^ 
60 percent of students by institutions, an increase from less than 10 percent in 
previous years. The focus of this validation was on adjuste'd gross income and taxes 
paid. The effectiveness of this increased validation in reducing error was reviewed 
and the following siiggestio^ns^re made: 

• Institutions fully complied with the requirement to validate in about 
83 percent of the cases, despite the burden they reported it imposed. Most 
of the remaining cases were partially validated. 

• Validation was effective in lowering error due to AG! and taxes paid, but as 
these were only the seventh and ninth highest causes of error, the overall 



effect on error was modest, though in the expected directio 
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TABLE 5-15 

FREQUENCY WITH WHICH STUDENTS AND PARENTS 
REPORTED DIFFICULTY WITH THE 
VALIDATION-^PROCESS, BY DEPENDENCY STATUS 



1 

. .a 



Dependent 
Students 



Independent 
Students 



Parents 
of Dependent 
Students 



Parents 
of Independent 
Students 



Percent^ige in sample reporting 

that they were selected for validation 



2&.5 



20.1 



19.9 



I 



Weighted Number Reporting §jelection for 
Validation 



Percentage repHjrting difficulty 
understanding instructions on what was 
needed to complete validation ' 



503 



7.2 



5A 



330 



37 



7.5 



Percentage reporting difficulty*^ 
• ohtaining^the documents needed to 
complete validation , 



8.8 



12.8 



21.6 
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TABLE 5-16 



FREQUENCY WITH WHICH INDEPENDENT STUBlE#FS AND PARENTS OF 
DEPENDENT STUDENTS REPORTED DIFFICULTY WITH THE VALIDATION 
PROCESS, BY TYPE AND CONTROL OF INSTITUTION 



Percentage reporting. selection 
for validation 



IVoprietary 



Less 
Than 

2 Year 2 Year 



15.6 



5.6 



Private 



2 Year * Year 
31.0 26.3 



Weighted number reporting 
selection for validation 



10 



IS 



115 



Percentage reporting difficulty 
with validation instructions 



10.0 



0.0 



5.6 



6.1 



Percentage reporting difficulty 
obtaining documents for validation 



0.0 



0.0 



5.6 



7.8 
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Validation had no effect on error due to incorrect reporting of dependency 
status, the largest soUrce of student error. New validation procedures 
would be required to reduce this error. 
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CHAPTER 6 
TRENDS IN PELL ERROR 

'I 

. The 1982-83 Pell Grant Quality Control Study is the third in a series of studies 
funded by the Office* of Student Financial Assistance. While certair^ differences exist 
across studies in methodology, training, thoroughness, and definitions, it is stUl useful 
to inspect the changes in error over time. These comparisons give program managers 
and policy makers an opportunity to examine program trends and the overall effects of 
program changes. In general, the following can be concluded: 

• The upward trend in error noted in 1980-81 has been turned around, as both 
student and institutional error dropped in 1982-83 ^ 

• The amount of overawards has decreased ,V(?|lile the amount of underawards 
has increased. , * 



6.1 NATURE OF POSSIBLE COMPARISONS AND LIMITATIONS 

In 1978-79, the first comprehensive Pell Grant (then Basic Grant) Quality 
Control Study provided an initial estimate of program-wide error. A second, more 
precise estimate w^s derived in 1980-81 and showed a disturbing upward trend in 
almost all components of error. As a result of both studies, sevet-al corrective actions 
were initiated. These included: 

More comprehensive computer edits of application data 
Increased validation of application data by financial aid administrators 
Data matches with other Federal agencies • 
Expanded use of program reviews and financial audits 

Redesign of the application form and instructions, including extensive field 
testing 

Printing of the Statement of Educational Purpose directly on the SAR 
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It is therefore desirable to compare the results of the l982-a3 Pell Grant Study 
to the earlier work to see if the corrective actions have been successful. In this 
chapter we provide comparisons of error across the three points in time. These 
comparisons are made for program-wide and component absolute, net, overaward, and 
underaward errors. However, due to differences in methodology, program regulations, 
and the environment, conclusions drawn from the comparisons should be made 
cautiously. Among the^considerations which influence the extent of comparability are 
the following: 

• 1978-79, data were collected in the late fall and early winter. In 1980- 
81 and again in 1982-83 data were collected in the late winter and early 
spring. ' . . . 

• In 1978-79, error computations were based qn comparisons o£ verified 
student data with expected disbursement figures. In 1980-81 and 1982-83, 
error computations were based on comparisons of verified student data 
with actual disbursement figures. 

' • Both the 1980-81 and 1982-83 studies collected secondary verification 
documents which were not collected in 1978-79. These documents were 
Internal Revenue Service copies of tax returns, documentation from 
financial institutfons on bank accounts, and tax assessor estimates of home 
values. 

• In both 1980-81 and 1982-83, data from institutions were collected by 
people well experienced in financial aid operations. In 1978-79, this was 
not the case. 

• Potential experimental bias was greatly reduced in 1982-83 from the 
previous studies through on-site selection of the recipient sample by the 
contractor.' ' 

• Social Security and VA Educational benefits were, for 1982-83 only, a 
direct adjustment to the Pell award, rather than an elerrient in the aid 
index computations. 

• For 1982-83, unlike prior years, estimated dependent student incorpe was 
used instead of the previous year's income when the estimate .was less than 
60 percent of the previous year's reported income. 

• For 1982-83, unlike prior y^rs, the Student Aid Index computation forrnula 
was changed, using a progressive "taxation" rate on available income, 
rather than a flat percentage. 

The maximum Pell Grant award was $1,600 in 1978-79, $1,750 in 1980-81 
. and $1,800 in 1982-83. ' 
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6.2 CHANGES IN OVERALL LEVELS OF ERROR 



Figure 6-1 depicts the trend in error from 1978-79 to 1982-83. Since the number 
Qf recipients varied across the years, it is most instructive to exanjine error per 
recipient. Average absolute error, net error, net student error, and net institutional 
error all dropped between 1980-^1 and 1982-83. Exdept for net student error per 
recipient, all other average error figures are at their lowest point in four years. 

In relative terms, absolute error per recipient, net error per recipient, net 
student error per recipient, and net institutional error per recipient are down 17 
percent, 2't percent, 9 percent, and percent respectively from the 1980-81 levels. 

Figure '6-2 looks at overawards and underawards. Both the percentage of 
recipients receiving overawards and overaward dollars per recipient showed a decline 
of slightly under 20 percent from 1980-81. A 15.6 percent drop in overaward dollars 
per recipient since 1978-79 was achieved with no drop in the percentage of recipients 
with overawards, indicating a drop in the average overaward error.' The percentage of 
recipients getting underawards and average underaward have been virtually unchanged 
over the three measurement periods. 

In an overall comparison, the frequency and degree of error has shown a 
significant decline from 1980-81 to 1982-83 after rising from 1978-79. One or more of 
the elements of corrective action and regulatory change have been successful. in 
reducing but not eliminating error. In the next section we examine how the various 
components of error have changed. ' *^ 

6.3 CHANGES IN ERROR COMPONENTS 

In this section we examine the trends in the components of student and 
institutional error from 1980-81 to 1982-83. The 1978-79 study was not as complete in 
its examination of the components of error, so that comparability is difficult. It 
should be noted that even more caution must be excercised in comparing components 
of error because these sub-aggregates are most . sensitive to changes in study 
methodology, program rules, and sampling error. 
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78-79 



80-81 



82-83 



ABSOLUTE ERROR 
PER RECIPIENT 



$168 


$170 




• / 










$129 
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$94 


$86 










$48 
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82 - 83 


78-79 


80-81 


82-83 



$120 








$77 














$39 


78-79 


80-81 


82-83 



NET ERROR 
PER RECIPIENT 



NET STUDENT ERROR 
PER RECIPIENT 



NET INSTITUTIONAL ERROR 
PER RECIPIENT 



FIGURE 6-1 
TRENDS IN PELL ERROR 
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1978 • 79 1980 • 81 1982 • 83 

OVERAWARD DOLLARS 
PER RECIPIENT 



1978-79 1980-81 1982-83 

UNOERAWARO DOLLARS 
P£R RECIPIENT 









I - 






50% 








41% 


41% 












1 18% j 21% 


21% 


1978-79 


1980 - 81 


1982 - 83 


1978 - 79 1980 - 81 


1982-83 



PERCENT OF RECIPIENTS 
WITH OVERAWARDS 



PERCENT OF RECIPIENTS 
WITH UNDERAWARDS 



FIGURE 6-2 

TRENDS IN OVERAWARDS AND UNDERAWARDS 
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6.3.1 Components of Student Error 

* 

Table 6-1 displays for 1980-81 and 1982-83 the net award error per recipient 
associated with each component of student error that exceeded $5 per recipient in 
1980-81. 

As already discussed in Chapter 5, there has been a 63 l>ercent reduction in error 
associated with parents'/independent students' adjusted gross income, much of which 
can be attributed to the increase in validation. Dependency status and household size 
continue to be error-prone application items, even though they experienced modest 
declines. Other nontaxable income error more than doubled from 1980-81 to'1982-83. 
A possible reason for this would be the , increase of transfer payments, especially 
unemployment compensation, between the two study years. 

■ ; ■ 

Other items showing more significant decreases include: 

• Home Equity 

• Net Student/Spouse Assets - 

• Net Student/Spouse Income 

• Investment Equity. 

The reasons for these decreases would include the possibility that home prices 
are now mqre stable than in earlier years and that the treatment of student/spouse 
income in the SAI computational formula has changed between the two years. 

6.3.2 Components of Institutional Error 
» 

Table 6-2 lists the absolute award error per recipient associated with each 
component of institutional error for 1980-81 and 1982-83. 

The most significant conclusion embodied in Table 6-2 is that the provision of 
the Statement of Educational Purpose directly on the 5AR was effective, reducing 
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• TABLE 6-1 

THE COMPONENTS OF STUDENT ERROR: 1980-81 and 1982-83 

J 



NET ERROR PER RECIPIENT 



1 









Percent 




1980-81 


1982-83 


Change 


!• DeDehdencv Status 


5 26. /Q 


$ 25.30 


-5% 


2. Other Nontaxable Income/AFDC* 


9.30 


19.90 


+ll'f% 


3, Household Size " « 


13.90 


I't.lO 




Number in Postsecondary Education 


5I9O 


9.60 


+63% 


5, Home Equity 


16.10 


7.70 


-52% 


6, Dependent. Student's (and SpcnfSe's) Assets 


11.00 


7.10 


-35% 


7. Parents'/Independent Studeitt's Adjusted 


16.10 


6.70 


-58% 


Gross Income . [ 








8, Dependent Student's (and Spouse's) Income 


18.20 


't.90 , 


\ -73% 


9, Real Estate/Investment Equity 


5.90 


0.70 


\88% 


All /Application Items 




86.00 


-9% 



*These two application items were, combined in 1980-81; thus we have added the'net 
errors for Other Nontaxable Income 4$ 19.20) and AFDC ($.70) for 1982-83 to provide a 
comparable figure. ^ ■ 



THE COMPONENTS OF INSTITUTIONAL ERROR; , 1980-81 Jhd 1982-83 



ABSOLUTE ERROR PER RECIPIENT 







1980-81 




Percent 






1982-83 


Change 


1. 


Program Eligibility Error* 


$ 10.30 


$ ^.50 


+2% 


2. 


No Statement of Educational Purpose 


35.90 




-89% 


3. 


No" Financial Aid Transcript 


35.90 


3i.50 




i^. 


Cost of Attendance Error 


26.60 . 


19.30 


-27% 


5. 


Enrollment Status Error 


39.90-. 


58.10 


+1^6% 


6. 


Calculation/ Accounting Error 


12.30 


22 AO 


+82% 


Institutional Error 


15^^.00 


127.00 

* 


-18% 



* Includes insufficient program length, less than half-time enrollment, nondegree 
programs, default on loan, not parent instifbtion, and unsatisfactory academic 
progress ' . . 
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error of this type by 89 percent. Missing Financial Aid Transcripts continue to be a 
problem of similar severity. 

Error associated w'ith enrollment status has increased; however, some of this 
increase^Tfi^ be attributable to improved measi/ement in the 1982-83 study. Enroll- 
ment status will continue to be a problem bec^use' of its highly volatile natUre. Study 
differences also may explain the change in calculation/accounting error.. . 



- CHAPTER 7 

VALIDITY OF RESULTS 

This chapter discusses three areas of critical importance ,to the findings, each of 
which reflects upon the validity of the results. Strictly speaking, these three areas-- 
strength of documentation, nopresponse, and experimpntal bias—are methodological 
issues that might be placed in.Volume 3 of this report. Procedures and Methods . Each, 
however, is.inciuffed in this volume for a pai'ticular reason, as followstt 

• ' Strength of documentation refers to our ability to collect data, from the 

rnultiple data sources, that assess the best value's for the varioiSs applica- 
^ tion items. .-The more reliable and credible 'the source, the stronger our 
documentation and ' our ^ ability to measure ..discrepancy and error 
accurately.. * ^ 

• Nonreponse bias refers to the effect of non respondents:; on the reliability 
and validity of the dita. Any differences .between respondents and non- 
rf spondents must be assessed and their effects, if any, taken into account 

' W weighting ,of .the^ f injiings. This weighting . is ' especiaHy important in 
[program-wide estimates of. error, since we are making estimates about the 
population from:our sciiti^ple.^ ' , * v - • ■ 

^ - ; • ^ Experimental bias refers to the-ef^fect of the data collection efforts on the 
findings. The issue is whether ^tq^ents .6r institutions participating in the 
. _ ' , study, change their behavior because of their participation,^ and whether 
. that behavipr affects our findings. :v-. / • ^ 

More detail on .spm4 of the issuies: raised in this chapter is included in Volume 3, but 
the major;.points fior aiH'understandi^^ of the findings are included here; 

7.1 STRENGTH OF DOCUMEN-TATlbN ^ ' ' ^ ; ■ ' ■ / : 

Our ability to measure" error in the Pell program is a direct fuhiikiQh , of our ^ 
ability to obtain. documentation or evidence regacding the verified or best value for a* 
particular item. : For institutional items, such as eligibility 'components, disbursement, 
cost of attendance^ and enrollment status, b^st values, were determined by a careful 
review of documents in various institutional offices by our data collectors. Since our 



.data collectors were experienced in financial aid and higher education, they were able 
to review documents in financial aid offices, registrars' offices, bursars' offices, and 
residential living offices efficiently and accurately'. - For student items, the process 
was considerably more complicated.- 

Determining the best value for student -application items involved personal 
interviews with students and their parents, reviews of the financial aid files at the 
/ institutions to find documentation presented to financial aid officers, and obtaining 
verification from third parties. The latter efforts involved requesting, reviewing, and 
coding more than 1^,000 certified tax returns provided by the regional IRS offices and 
hundreds of records on home assets and financial assets from tax assessors and 
financial institutions, respectively. 

For each application item, there were many sources of information and the 
potential for as many different values for an item as there were sources. The first 
task was to assign^ priorities to all of the possible sources according to the: perceived 
strength of documentation for each. This involved both objective criteria (e.g., a 
certified copy of a tax return from the IRS is a more reliable or stronger source of 
documentation than either a photocopy of a worksheet used in preparation of the tax 
return or a statement provided by a professional tax preparer) and professional 
judgment. Judgment was required in areas of greater subtlety. Which, for example, is 
a better source of documentation for home value: an appraisal from a real estate 
office, a property insurance form, or a rhortgage statement? According to our 
priorities these are listed in order, beginning with the strongest^ but the mortgage 
statement is acceptable only if the house was purchased within the last three years. 

Since different sources were provided as documentation according to dependency 
status, we first determined the best depencigncy status, along with items that were 
needed to determine documentation for other items. It was-necessary, for example, to 
know the best value for both marital status and tax filing status before determining 
AGI. (If the best marital status^ was divorced, but the most recent tax return was^ 
married filed jointly, that tax retlfrn cannot be used unlesstli^ere was evidence of the 
incAme portions attributable to each person.) Because 'of differences in possible 
documentatioil^ and.^Peli regulations^ best values had to t>ej separately detfl^ined for 
indepeii^ent students, dependent students, and parents of dependent students. 

':■ . ■ : . ■ I,. ■ 

■ ■■■ - .. . /'c. 
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Table 7-1 summarizes the strength of the sources of documentation used in 
determining best application values for major application items. The mean, standard 
deviation, and weakest acceptable documentation priority are given. The weakest 
acceptable priority is not necessarily indicative of the number of possible items of 
documentation, since any one priority may have multiple sources tied for a given rank. 
If, however, those sources were the same item (e.g., tax return worksheet) from two 
different places (parent interview and student's financial aid file), we always gave the 
documentjition from the^ersonal interview a higher priority. 

Together the mean and standard deviation sh6w the strength of the documenta- 
tion availably. For AGI, the mean of 1.6^ placed it just below priority 1, out of 17. 
Priority one was a certified tax return from the IRS (Line 31 on the 10^0 or Line 10 on 
the lO^OA) from which ar^ reported' College Work-Study earnings (which are. not 
considered part of AGI for the Pell) were subtracted. Priority two was a copy or 
worksheet of the same tax return shown to the student or parent interviewer.- The 
narrow standard deviation shows that the priorities used in the majority of cases were 
tightly clustered near the strongest sources. 

^ The number of cases with documentation and the percQ^it of cases with 
documentation are provided next. Items with low percentages of documentation 
reported were ''typically those for which very few people had the income or asset in 
question^ It was nearly impossible, for example, to document that one had not 
received Aid to Families with Dependent Children or did not own. any real estate or 
have any other investments. The last column presents the percentage of cases whose 
best value was discrepant from that on the application. Application values within $2 
of the best value were not considered discrepant, to be consistent with the definition 
used for error. For non-dollar items, any difference was considered to be a 
discrepancy. 

On the whole, documentation for most application items was strongs as 
evidenced by low means and narrow standard deviations. Only home value appeared to 
have considerable variation in the documentation provided, perhaps reflecting tha|lta^a,- 
assessors* offices were contacted for only 25% of reported home owners. The 
proportion of cases withadiscrepant values varied widely, but tended to be highest for 
thos? items with fewer persons having th^ income or asset in question. Notable 



TABLE 7-1 

PRESENCE AND STRENGTH OF DOCUMENTATION FOR SELECTED APPUCATION ITEMS 



Strength of Documentation Cases with Documentation 







Standard 








Percent 




Mean 


Deviation 


Acceptable 


N 






Adjusted Gross income 


1.64 


1.52 


17 


2,486 


78.3 


23.2 


U.S. Taxes Paid 


1.48 


1-19 


12 


2,444 


76.9 


2«|3 


Appiicant's/Father's Income 


4.28 


3.12 


21 


1,662 


52.3 


44.5 


Spouse's/Mother's Income 


4.51 • 


3.39 


21 


1,339 


42.2 


34.0 


AFDC 


1.54 


.70 


4 


154 


4.8 


63.6 


Other Nontaxable Income ^ 


1.02 


.14 " 


2 , 


2,840 


89.4 


35.0 


Child Support ^ 


— 





1 


217 


6.8 


e 


Other Welfare ^ 


— 


— 


1 


93 


2.9 


e 


Non-Educationai VA Benefits ^ 


— 


— 


1 


71 


2.2 


e 


Unemployment Compensation ^ 


1.02 


.14 


2 


2,119 


66.7 


• e 


interest /Dividend Exclusion ^ " 


1.02 


.14 


2 


2,121 


66.8 


e 


Number of Exemptions 
Household Size ' 


1.49 


1.18 


15 


2,451 


77.2 


8.6 




— 


— 


3,144 


99.0 


24.2 


Number in Postsecondary Education^' 


-- 


— 


— 


3,167 


99.7 


14.6 


MedicSl/Dental Expenses ' f 


3.43 


2.38 


9 


" 1,055 


33.2 


71.5 


Elementary/ Secondary Tuition 


1.08 


.39 


3 


106 


3.3 


69.8 


Cash/Savings/Checking 


2.32 


1.23 


4 (DS) 


1,324 


41.7 


46.8^ 








5 (IS; 








Home Value 


6.56 


3.15 


9 


1,609 


50.7 


42.2 


Home Debt 


1.25 


.83 


6 


678 


21.4 


78.5 


Real Estate/Investment Value 


1.38 


1.10 


5 


107 


3.4 


89.7 


Real Estate/Investment Debt ^ ' 


1.72 


1.21 


5 


61 


1.9 


62.3 


Farm Equity /Farm Value 


1.12 . 


.59 


4 


26 


0.8 


e 


Farm Debt ^ ^ 


1.18 


.72 


4 


34 


1.1 


e 


Business Value 
Business Debt' 


1.07 


.38 


4 


54 


1.7 


e 


1.33 


.73 


4 


46 


1.4 


e 


Parent's Social Security Benefits^ 


1.26 


.48 


4 


155 


7.7 


88.4 


Expected Social Security Educational Benefits 


1.89 


.91 


4 


110 


3.5 


5.4^ 


Expected VA Educational Benefits 


1.85 


1.20 


4 


48 


1.5 


95.8 


Student Marital Status 








3,168 


99.8 


9.6 


Parent Marital Status *j . 








2,004 


99.2 


3.5 


Dependent Student's (Spouse's) Income .'j^ 


1.13 


.36 


3 


1,322 


65.4 


56.1^ 


Dependent Student's (Spouie's) Assets 


1.00 


0- 


2 


2,018 


99.9 


58.8^ 



* Strongest sources are those with lowest numbers indicating highest priority. Comparisons across items cannot be directly made 
^ since a priority of, for example, 5 on one item may reflect a better source of documentation than a 3 on another. 

Includes some partially undocumented values to avoid the problem of failing to report one of the several contributing sources to 

this item because of lack of documentation. 

Q 

^ 'For these components of other nontaxable income (as well as others not shown here) undocumented values were accepted. 

These components of crther nontaxable income could be documented for tax filers. For those who did not file a tax return, or for 
^ whom we received no tax return, undocumented values were accepted* 

^ Discrepancy could not be calculated on these W>components since the application requests total figures only. 

It is not possible to document these items «nce application questions were estimates for upcoming year; thus, cases shown 
represent all responses^ 

^ Includes undocument^. values to avoid the problem of reporting a documented value but no debt because of lack of documentation. 

Discrepant cases based on total of documented and undocumented cases. 

Largely undocumented; cases shown represent all responses. 
^ Percentage with documentation based on number of verified dependent students. 
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exceptions to this tendency were applicant's/father's income, sg^se's/mother's 
income, other nontaxable income, and home value. 



7.2 NONRESPONSE BIAS 

Despite efforts to assure a high level of response during the course of the. data 
collection, there were some cases that were functionally considered nonrespondents. 
The purpose of this section is to assess any bias that might have been introduced by 
nonrespondents and to detail our procedures, for adjusting the sample for non- 
respondents. 

7.2.1 Overall Response Rates and Definitions of Nonresponse 

Different definitions of nonresponse can be used for each of two different phases 
of the study. For interviewing, the response rate was simpiy the number of cases 
interviewed in a given group. Thu$^ 95 percent of the paired student and parent 
interviews for dependent students and student interviews only for independent students 
were successfully completed, after cases sampled in error had been removed from the 
sample. Yet this figure did not directly relate to the respons^e rate for analytical 
purposes, since different criteria applied for a complete case. 

^ For analysis, a case was considered complete if, for a dependent student, there 
was a student interview, a full parent interview, and a student record abstract. For 
independent students there must have been a student interview and a student record 
abstract.^ The requirement for a full parent interview presented unique problems for 
dependency status switchers. If,-at the time of the interview, the student's change in 
status from independent to dependent was not known, a partial parent interview was 
scheduled. This interview included questions to establish or verify dependency -gtatus 
and household size, but stopped before a probe of income and assets was conducted. 
This was because parents of independent students were under no obligation to 
participate and we were sensitive to their concerns for privacy. If a dependency 
status change was apparent, an effort was made to complete a lull parent interview. 
Often, however, the dependency status change was not evident until later," and an 
otherwise complete case became a functional nonresporident because no income or 
asset information was available from the parent of the now;-dependent student. 
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These and other response probl^s reduced the useful resporfee rate for the study 
to over 86 percent, as measured by the ratio of complete cases to the sum of 
incomplete and complete cases. This is illustrated in Table 7-2. 

7.2.2 Assessment of Nonresponse 

The next stage in the process for nopresponse was to assess any differences that 
might exist between respondents and nonrespondents that could iiWicate nonresponse 
bias. To compare respondents and nonrespondents it was necessary to select data that 
were readily available but indicative of error for both groups. Since nonrespondents, 
by definition, were those for whom certain information was not available, the pool of 
data for comparison was limited to that in existing records. Four items that provided 
a good basis for comparison because of their likely reflection of error were- SAI, 
^scheduled award, expected disbursement, and number of transactions. These were 
available from recent records of the Pell Grant Disbursement System for both 
respondents and nonrespondents. 

A series of multiple regression analyses were conducted for each of these 
dependent variables, with respondent status, dependency status, and selection for 
validation as additional regressors. The results revealed a need for nonresponse 
adjustment, as depicted in Table 7-3. There were significant predicted main effects 
for respondent status on scheduled aw^ird, expected disbursement, and number of 
transactions, with respondents having smaller awards and fewer transactions. There 
was no significant main effect for SAI. 

7.2.3 Nonresponse Adjustment 

Since response rates may differ according to student and institutional 
characteristics, separate adjustments were computed for each of numerous possible 
response groups into which individual recipients fell. We adjusted four characteristics 
of Regular Disbursement System (RDS) recipients: 

^ ■ 

Dependency Status (2 categories) - Dependent • 

- Independent 
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TABLE 7-2 



RESPONSES AND RESPONSE RATE FOR ANALYTIC PURPOSES 
/ 

... / 



Number Sampled < ^,109 

Number Sampled In Error ^22^ 
Net Sample 3,687 
Nonrespondents 512^ 
Respondents 3,175 

Response Rate = 3,175/3,687 = .861 



Sampled in error Includes students not receiving a Pell at the Institution indicated, 
students who dropped out of school at such a time that the full amount of any 
disbursements made was completely repaid, and dependent studeats whose parents 
had died or were out of the country. 

Nonrespondents include those without required Interviews and those whose computed 
applicant record from the central processor could not be obtained because of 
damaged records on the processor's tape or failure to match Social Security 
numbers. 
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TABLE 7-3 *^ 
REGRESSION ANALYSES TO ASSESS FOR NONRESPONSE BIAS 

Model Ix Dependent Variable: SAI 

Independent Variables: 

Main Effect - Respondent Status (RESP) 

Interaction Effect - Respondent Status by Dependency Status (DEPS) 
Interaction Effect - Respondent Status by Selection for Validation 
(SEL) 

Results: R^ = .116, £ = ,001 (df = 5, 357*) 

Sources of Variation: df F £ 

RESP 1 ' .21 .6<f77 

RESP X DEPS 2 20*. 62 .0001* 

RESP X SEL 2 16.79 .0001* 

Least Square Means for SAI : 
Respondents 323.3 
Nonrespondents 312.5 

Model 2: Dependent Variable: Scheduled Award 

Independent Variables: (Same as Model 1) 

Results: R^ 3 .03*, £ = .0001 (df s 5, 3561) 

* Sources of Variation : df F £ 

RESP r 6. 12 .013*» 

RESP X DEPS 2 52.5* .0001* 

RESP X SEL 2 3.55 .0287* 

Least Square Means for Scheduled Award 
Respondents 1X35.* 
Nonrespondents 1195.5 

Model 3: Dependent«Variablet Expected Disbursement 

l/vdependent Variables: (Same as Model 1) 

Results: R^' » .022, £ x .0001 (df = 5, 3561) ♦ 



RESP 

RESP X DEPS 
RESP X SEL 



df 


F 


2 




1 


7.01 


.0081 




2 


3*. 13 


.0001* 




2 


l.*l 


.2*5* 





Least Square Means for Expected Disbursement 
Respondents 1076.6 
Nonrespondents 11*2.8 

Model *: Dependent Variable: Number of Transactions 

Independent Variables: (Same as Model 1) 

Results: R^ = .052, £ = .0001 (df = 5, 3591) 



RESP 

RESP X DEPS 
RESP X SEL 



df 


F 




1 


351*2 


.0001* 


2 


79.97 


.0001* 


2 


*.95 


.0071* 



Least Square Means for Number of Transactions 
Respondents l.*6 
Nonrespondents 1.72 



♦Statistically significant. 
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Type and Control of Institution 
(6 categories) 



Validation Status (2 categories) 
SAI (5 categories) 



Public 2-3 years 
Public ^ years or more 
Private 2-3 years 
Private ^ years or more 
Proprietary less than 2 years 
Proprietary 2-3 years 

Selected for Validation 
Not Selected for Validation 

0 

1 - ^^00 
.^01 - 800 
801 - 1,200 
1,201 . 1,600 



In addition, we adjusted for three other groups: 



Alternate Disbui^iement System (ADS) Recipients 

Independent to Dependent Dependency Status Switchers 
\ 

Dependent to Independent Dependency Status Switchers 



^Adjustment for any cell was based on the reponse rate, r.: 



r. = 
1 



number completed 



number complete + number incomplete j 



If the denominator of any rj was less than 25 that group was combined with an 
adjacent cell orjzells. Jhe first choice for combining cells was type and control of 
institution, followed by validation status. 

Inability to assign all cases to a cell necessitated an additional group for persons 
with missing application values on any of the four characteristics upon which 
assignment was based. , After combining groups with small denominators, ^9 cells were 
created. For each cell the nonreponse adjustment weight was the inverse of the 
response rate or l/rj. The resulting cells and nonresponse adjustments are shown in 
Table 7-^. 

These nonresponse adjustment weights were used in all tables dealing with error 
to produce means, program-wide estimates, and percentage of cases with error, unless 
otherwise noted. 
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TABLE 7-4 

NONRESPONSE ADJUSTMENT VEIGHT BY SELECTION CHARACTERISTICS 



1 
2 
3 

5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 

42 

43 

44 

45 

46 

47 

48 

49 



Group Prop 1 Prop 2 



X > 



X 
X 

X 
X 
X 
X 



X 
X 
X 
X 
*X 
X 



and Control 



Pub 2 p?rr 



VaUdation Selection 

Not Dependency SAI^ 

Pub 4 Selected Selected Status' Group^ 



X 
X 

X 
X 
X 



X 
X 

X 
X 
X 
X 



X 
X 
X 
X 
X 
X 



X 
X 
X 
X 
X 
X 



X 
X 

X 
X 
X 



X 
X 
X 
X 
X 
X 



X 
X 



X 
X 



a 
b 



D = Dependent 
SAI Group 



I = Independent 



X 

X 

X 

X 
X 

X 
X 
X 
X 



Independent to Dependent Status Switchers 
Dependent to Independent Status Switchers 
ADS Recipients 
Unassigned c 



X 
X 
X 
X 
X 
X 
X 



X 
X 
X 
X 



X 
X 
X 



X 
X 
X 



X 
X 
X 



X 



X 
X 
X 

X 



X 
X 

X 

X 

X 



X 
X 
X 
X 



X 
X 
X 



X 
X 



X 
X 



X 
X 



X 
. X 
X 
X 
X 



X 

X 

X 
X 



D 
D 
D 
D 
D 
D 
D 

b 

D 
D 
D 
D 
D 
D 
D 
D 
D 
D 
D 
D 
D 
D 
D 
D 
D 
D 
D 
D 



Number 
of 

Complete 
and 



Number 
of NonrespORse 
incomplete Complete Adjustmon. 
Cases Cases Weight^ 



0 


29 


27 


1.0741 


0 


84 


70 


1.2000 


0 


74 


67 


1.1045 


0 


138 


124 


1.1129 


0 


46 


43 


1.0698 


0 


95 


83 


1.1446 


0 


45 


39 


1.1538 


0 


79 


67 


1.1791 




78 


73 


1.0685 




75 


69 


1.0870 




165 


151 


1.0927 




61 


57 


1.0702 


} 


33 


26 


1.2692 




73 


68 


1.0735 




44 


42 


1.0476 


2 


77 


71 


l.M^5 


2 


153 


142 


1.0775 


2 


41 


36 


1.1389 


2 


63 


54 


1.1667 


3 


49 


46 


1.0652 


3 


67 


61 


.1.0984 


3 


144 


135 


1.0667 


3 


30 


26 


1.1538 


3 


54 


47 


1.1489 




64 


60 


U0667 
1.0256 


** 


\20 


- 117 


4 


36 


32 


1.1250 


4 


37 


^7 


1.3704 


0 


30 


27 


1.1111 


0 


121 


106 


1.1415 


0 


50 


48 


1.0417 


0 


317 


294 


1.0782 


0 


39 


34 


1.1471 


0 


39 


30 


' 1.3000 


0 


141 


116 


1.2155 


0 


45 


35 


1.2857 


0 


U7 


96 


1.2188 


1 


25 


23 


1.0870 


1 


45 


44 


1.0227 


1 


54 


43 


1.2558 


2 


54 


.50 


1.0800 


2 


43 


34 


1.2647 


3 


39' 


36 


1.0833 


3 


48 


37 


1.2973 


4 


49 


34 . 


1.4412 




184 


66 


2.7879 




13 


11 


1.1818 




23 


16 


1.4375 




70 


197 


1.0000 



0 = 0 

1 = 1 - 400 . 

2 = 401 -800 / 

3 = 801 - 1200 

4 = J201 - 1600 

l^n"^^ul:^:r°^StZ^Zl^ '° '''' '"i-»l"8 characteris-tlcs. a nonreponse adjustment ol 1 wa, used. Thl, lead, to 

Mean nonresponse adjustn\ent weight = 1.1501 
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7.2.4 Sensitivity Analysis Using Alternate Nonresponse Assumptions 

f 

The error adjustment procedures described for nonrespondents provided a means 
for arriving, at error estimates for the population. WhUe we are confident that these 

■ procedures yielded reasonable estimates, we recognize that using other assumptions 
for nonrespondents would yield different error estimates. In this section we predict 
some error estimates for net case error and net total error based on a series of simple 
assumptions about nonrespondents. These estimates can be compared to the estimates 

"^bbtained using the nonrespondent adjustment weights described above. This approach 
can be called sensitivity analysis because it shows the sensitivity of error estimates to 
varying nonresponse assumptions. ' . ' 

\ 

A series of five alternate assumptions regarding nonrespondents were selected 
for this sensitivity analysis. These assumptions were based on their reasonableness, 
determined in part from a similar analysis conducted for the 1980-8.1 data. Table 7-5 
shows net error estimates using the following alternative assumptions for. mean error 
per recipient: 

Mean error for nonrespondents w^equal to that of respondents. 

Mean error for nonrespondents was equal to the 95th percentile of mean 
error for respondents. t 

. ■ ^ ■ 

Mean error for nonrespondents was equal to the 90th percentile of mean 
error for respondents. ^ 

Mean error for nonrespondents was equal to the 75th. percentile of mean 
error for respondents. >. ' v - . 

' , • ' - •.'^^ 

Mean error for nonrespondents was equal to the 50th percentile (media^n) of 
mean error for respondents. 



The first and the last assumptions used different measures of central tendency, the 
mean and the median for respondents, as the basis for determining nonrespondent error 
and program-wide estimates. The other estimates.were based on the assumption that 
nonrespondent error was equal to that of a selected percentile of respondent error. 

The formula for the mean error used in these sensitivity analyses relies upon the 
proportion of respondents (.861) and rfenrespondents (.139) derived from the data in 
Table 7-2. Thus: 
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Mean error = . (r) (mean error for respondents) + (l-r> (assumed^ error for 
nonrespondent(s), where 

r = proportion of respondents in the net sample 

* 

The program-wide estimate of error was simply the mean error multipUed by the 
estimated 2.53 million Pell recipients. 

An inspection of the results in Table 7-5 reveals that the estimates derived from 
the nonresponse adjustment weights (shown in the last line of Table 7-5, as taken from 
Table 2-1) fall in the middle range of the estimates repprted using the alternative 
assumptions. 

7.3 E)^RIMENTAL BIAS 

: Experimental bias refers to the effects of the treatment (in this case, inter'^ 
viewing students and their parents, interviewing financial aid administrators, artd 
reviewing student financial aid files) on the outcome (in this case error^<>^ any "behavior 
^ of. students, parents, or financial aid administrators that may affect error). Experi- 
mental bias is a potential problem in any type of study where people can react in ways 
that influence the outcome. . , 

In the conduct of any study involving extensive intervention, experimental bias 
cannot be eliminated. Considering the attention given to the findings of Stage One. 
within the financial aid community and tHe response to the imposition of more 
extensive validation for 1982-«5, institutions notified in late 1982 of their selection 
for Stage Three may have reacted, to the extent that differences exist between 
sampled and nonsampled students on ^variables used for calculating student and 
insti'tutional error, experimental bias, can be assess^^^^^^ 

To assess the extent ^to which institutions treatei^ sampled students differently 
than nonsampled students, we collected information on a control group" sample of 
students. This institutional control group (ICG) consisted of 611 nonsampled ^eSipi^nts 
at. 263 pf the sampled institutions. Neither the students nor the institutions were 
aware of this control group in advance of the institutibhal vijit^ The data coliected 
were restricted to items readily available in the student's financial aid file 
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TABLE 7-5 

ERROR ESTIMATES USING ALTERNATIVE 
ASSUMPTIONS FOR NONRESPONDENTS 



Net Case Error 



Assumption 
for Nonrespondents ^ 

Respondents and nonrespondents 
have equal eruor 

Nonrespondents have error equal 
to percentile of respondent 
error: . 

. 95th percentile 
90th percentile 
75th percentile 
50th percentile 

rslonresponse adjustment weights 
used for nonrespondents.^ (method 
used elsewhere in this report) 



Program-Wide 
Estimate 
(SMiUions). 



288 



590 
*51 
305 
2*7 



326 



Mean^ 
Erroi;''per 
Riecipierit 



,233 
98 



129 



Mean 
Nonrespondent 
Error per 
Recipient 



ipi( 



97it 
579 
165 
0 



Net Total Error 



Program-Wide 
Estimate 
($ Millions) 



283 



657 

m 

260 
2ttk 



Mean 
Error per 
Recipient 



112 



260 

160 : 

103 
96 



125 



Mean 
Nonrespondent 
Error per 
Recipient 



ipii 



112. 



1,175^ 
*56 
*5 
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: aivi t^s cou^^ without the stuttent or Jnstit^tioh' .beij^ .. ; 

behaVior;y: Y^l^^ variables coul4'.pen be comp^V.edJ;ior^|^^^^ ■ 

s%J^nt^^:^nd -the blind sample ors:ontrol gr'^p. If the siai<^^hts -alterbd i^fieir behavior ' 
bedause.of . selection for the study, we would expect thern :to .haVei low^^^ more" ' 

transactions, ancj lower expected disbursements than the centV^l Igrpup stud Who ' 
weriq not aware ol-the data collection. 1: ■ \ ^' ''' • " 



Table 7-6 compares the average values for seven selected, items for sarnpled and 
\,IGG students. For only one item. Social Security educational? bene^^^^^^ the- 
• difference greater than four percent. Generally, the dif ferences Wre' bj^tween one 
and two percent. Therefore, the results support the conclusion that V^th^.: sdnipled 
institutions treated sampled students no differently than other aid recipients. 




Informi^j^^^S also collected on the frequency of institutional eligibility errors. 
Table 7-7 displays the frequency of the eleven types of institutional eligibility error 
for sampled and ICG students. If schools treated sampled students more c'afefully in 
order to appear less error-prone, we would expect higher error rates for the control 
; group. -While the rates were higher for some items, the likelihoods of. these errors 
w^r^;'so low that comparisons at the detailed level can only be made with great' 
cautipn J, ; Overall, the incidence of elig^ibility': error was lo^yer .-for tfcife 'control group, 
•. Which; Wats; :contrary to what would ;. be ejflSeCted if institutions wfe^ altering their 
behavior and creating experimental bias. 
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A final assessment of experimental bias involved the use of appreciably more 
control group'data. This Wasyacpomplished in several steps:- ' , s"^ ':r . 



• .Drawing a random sample of 20,000 'appli<;artts "from;. the Computed 
Applicartt Record (CAR) maintained by the Pel^central processor 

>• : ,: this random sample with the Pell Recipient History File to 

excludife norirecipients and obtain the institution attended, scheduled aw:ard, 
and 'dxpected disbursement 

• Separating the random recipient file into two groups: ' 

; 1>3^3 additional students at sampled institutions to expand the size of 
v v the ICG to a total of 1,92^ (expanded ICG) • ^ ^ . 

— • . ;6,dl3, students at nonsampled institutions to use as an additional 
\ ■ ' ^ controi.group (CAR-CGK ; " 
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COMPARISON OF SELECTED ITEMS TO 
ASSESS EXPERIMENTAL BIAS 



Sampled Students 



Control Group. StudentsI 



Item 

SAI 

Scheduled Award 

Expected Disbursement 

Social Security Educational 
Benefits 

VA Educational Benefits 

Cost of Attendance 

Transaction Number 



Number.of 
Casgs^ 

3,742 
3,452 
3,416 
312 

59 
3,451 
3,759 



Average 
, Value 

366.82 
1,126.94 
1,073.01- 
1,430.31 

. 2,980.75 
3,521.09 
1.55 



Number of 
Cases 

611 « 
/565 
569 
54 

8 

568 
608 



Average 
Value 

352.97 

l,58p:22 

2,955.00 
3,465.71 
1.5^ 



The rtumber of cas&^r sampled students is higher in this table than elsewhere in the report because of the 
less d^mani^ift^ data requirements of this analysis. 
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TABLE 7-7 

COMPARISON OF INSTITUTIONAL EWGIBtLITY PROGRAM ERRORS TO 

ASSESS EXPERIMENTAL BIAS 





Samp 
6 


led Students 
4 = 3786) 

% 


Control 
>Group Students 

(N = 6U) 

-v... N'. • 


NoSARinfile 




.11 


,0 


0 


Invalid SAR 


15 




2 


.32 


Statement of Educational Purpose 
not signed 


20 


.53 




.65 


Less than half-time enrollment ^ 
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0 


No Financial Aid Transcript 
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3.57 
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3 


.08 


_0 
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Total Eligibility Errorsi : - 


23't 


6. IS 


30 


*.91 



The number of cases for sampled students is higher in this table than elsewhere in 
the^f eport because of the less demanding data requirements of this analysis. 
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Experimental bias within institutions refers to the differential treatment of 
sampled students and nonsapnpled students by the sampled institutions. It was assessed 
separately for certainty institutions (those 3^ institutions in the study whose large 
numbers of Pell recipients made the institution certain to be included in the 
institutional sample) and noncertainty institutions by comparing the mean SAI, number 
of transactions, scheduled award, and expected disbursement for sampled students and 
nonsampled (expanded ICG) students- The results are shown in Table 7-8, For both 
certainty institutions and noncertainty ihstitutions there were no significant 
differences in the expected direction between the sampled students and the control 
group on number of transactions, SAI, scheduled award, and expected disbursement, 
(If experimental bias existed, students at sampled schools would^ave a significantly 
higher number of transactions and SAI and significantly lower values for scheduled 
award and expected disbursement.) Thus, there is no reason to believe that anything in 
the study influenced the sampled students to change their behavior in any way that 
seriously affected the Pell Grant. 

Experimental bias across institutions refers to differential treatment of 
nonsampled students at sampled institutions (expanded* ICG) and nonsampled students 
at nonsampled institutions (GAR-CG). Meaningful differences might indicate that 
selected institutions tightened their review procedures or persuaded the students to 
institute changes. As Table 7-8 shows,- there were no significant differences in the 
expected direction in number of transactions or SAI between the groups, suggesting no 
changed student behavior. Students at sampled institutions had statistically 
significant lower scheduled awards and expected disbursements which would be 
consistent with the expected direction of any bias. However,* the small magnitude of 
these differences, the extremely large numbers involved, and the findings on SAI 
suggest that the differences were not meaningful. Thus, there wjas also no evidence of 
exRjerimental bias across institutions. 

The results confirm that there was no systematic experimental bias brought 
about by the conduct of the interviews and record abstracts. The procedures used 
were methodologically sound and posed no threat to the validity of the results. 
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TABLE 7-8 

DIFFERENCES BETWEEN MEANS OF KEY MEASURES FOR 
ASSESSING EXPERIMENTAL BIAS 



Type of Experimental 
Bias and School 

Within Institution 
for Certainty Schools 



Mean 
Measures 



I 



Within Institution 

for Noncertainty Schools 



Number of transactions 
SAI 

Scheduled award 
Expected disbursement 



Number of transactions 
SAI 

Scheduled award 
Expected disbursement 



Mean 
Measures 



Across Institution 

for Noncertainty Schools** 



Number of transactions 
SAI 

Scheduled award 
Expected disbursement 



Sampled Students 
at Sampled Schools 

(N = 273) 
. 1.55 
338 
1,153 
1,097 



(N = 2,886) 

r 

1.50 

377 
1,097 

Nonsampled Students 
at Sampled Schools 

1^ '(N= 1,338) 

360 
1,081 
1,003 



Nonsampled Students 
at Sampled Schools 

(N = 586) 
IAS 
31^9 

i,m 

1,020 



. (N = 1,338) 

1.5it r 
360^ 
1,081 
1,003 

Nonsampled Students 
a t Nonsampled Schoo ls 

{N = 6,013) 
1.50 
308 
1,173 
1,032 



Statistical Procedure 



t 


.df 


* P 


.90 


857 


>.05 


-.32 


857v- 


>.05 


.29 


857 


>^05 


2.33 


857 




-1.25 


1^,223 


>.05 


1.12 


i*,i23 


>.05 




It, 223 


>.05 


2.62 


1^.22^ 


>.05 


Statistical Procedure 


t 


df 


. P 


1.32 


7,350 


>05 


3.70 


7,350 


>.05 



-6.3tt 
-1.88 



7,350 



<05* 
<.05* 




♦Statistically significant using a one-tailed test.. 
**There can be no across institution measure of bias 
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for certainty schools because all certainty schools are sampled. 

{ 
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APPENDIX 
ERROR DEFINITIONS AND EQUATIONS 



• A central concern was the development of appropriate definitions for error in 
the Pell Grant program. In the broadest sense, error is simply the discrepancy 
between the dollar amount of Pejl awards actually disbursed and the correct amount 
that shoulcl have beien <iisbursed. the determination of these correct amounts is 
based upon the "best values" for each item as collected from the various sources. 

Great attention was directed toward the successful collection of information 
and documentation that could be used to verify the values reported by applicants 
(both students and their paren?^i)^ used by institutions in the determination of the 
• Pell award. The- methodology-(*fi(th^ study is based on the assumption that in the. 
absence of information to corvtr^^d^t'a reported figure that figure must be presumed 
correct. In oth(Sr words, if no documentation of error could be identified for any 
item in any caie, error wa^ presuril||^jjt>;be zero. This assumption must he made in"^ 
confirmatory stti^ oi^^js sa^rVsin^e ip the majority of cases it is Ukely that Uttle 
or no error exists. T5^!l=esult is''aj|nd^ncy toward underestimatipn of error. This 
tendency is counlipc^;|ji^^^^^ the efforts to secure verifying informa- 

tion from '"^uitipl€^<^Ata increased the chance that error wouid 

be found and decre^s^^^ must be omitted from the sampie 

because of the inabilftir) t6 -obtain documentation. 

' v-v:'; ^, I ^ \ 

The deter minati*i.of%|ie',Pe^^ on a somewhat complex formula. 

The crucial elements ^^{>i^ft '5^ the assertion that 

eligibility requirements h^>Fe>;'b^:e(ht^^ the award for less than full- 

time, full-year enrollmeht.:^tat'usi^;i^ .4]pp^^^ resulting amount is called the 

Unajusted Expectecy^DisbuVs^ment-fUgii^^^^^ is calculated, it is checked 

I ■ ■■■■ i.A- ■ 

against another award formu%VtbSf^^^^l^^ Disbursement (MED).. The 

MED takes into account t\yb Itemj, ;^^ benefits arid "Veterans 



Administration educational benefits, that are excliSded from the UED formula. Tlie 
formula for the MED is: 

MED =Cost of Attendance - SAI - Social Security Educational Benefits - 
1/3 VA Educational Benefits, pro-rated for less than full time, 
full-year enrollment status, if applicable. — 

The MED sets a maximum. Thus, the lower of UED or MED is called thie Adjusted 
Expected Disbursement (AED). 

' i ' / ' _ 

The equations which follow indicate the sources of the values that/ will be used 
in each error computation. The algebraic computation of AED will follow ED 
regulations. Each formula uses a series of standard abbreviations. These are: 

AED - Adjusted Expected Disbursement, with data values used in it^ computa- 
tion shown parenthetically 

AD Actual Disbursement, the sum of disbursements already made by the 
institution and disbursements planned by the institution' 

SAI - Student Aid Index - 

COST - Cost of Attendance 

ENROLL - Enrollment Status 

ELIG - Categorical Eligibility 

J. EB - Educational Benefits 

^f^v^^^-^i'^ last four abbreviations (SAI, COST, ENROLL, ELIG) are always shown 
:^^v % ' a subscript indicating which of the multiple values is to be used for that 

^-^fci^^^^V'l rf These subscripts are: O ' \ ' 

t%:0!^-''^ ^ - Vajlue from SAR/CAR jOr SRA (as transcribed from Section 3 of file 

■ ^ SAR) used by institution for determining award 

* - Best value as determined by Advanced Technology according to 
merge priorities. ^ * 

For example, the expression AED (SAI*, COSTb, ENROLL*, ELIGb, EB*) 
would mean the value oi the adjusted expected disbursement calculated based 
on the "best" values for SAI, ENROLL, and EB and the SAR values for COST, 
and ELIG. 
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For purposes of clarity, 'these error equations are shown for total error, 
student error atji its major parts, and institutional error and its major parts. 



The error equations are: 



CASE ERROR 

.AD - Aed (sai*, cost*, enroll*, ELIG*, EB*) 

STUDENT ERROR 
Overall 



At D D Alb COST* ENROLL*, ELIG*, EBb) 
- AED (SAI*, COST*, ENROLL*, ELIG*, EB*) 

Components and Items of Student Error 

AED (SAIb, COSTb, ENROLLb, ELIGb, EBb) 

- AED (SAIb/», COSTb, ENROLLb, ELIGb, EBb/»), 

where SAIb/* and EBb/* are calculated using b values for all elements of 
student error except the element(s) to be icientified for error, which 
use(s) * values 

INSTITUTIONAL ERROR 

Overall 

AD - AED (SAIb, COST*, ENROLL*, ELIG*, EBb) 
Eligibility Errorl 

AED (SAIb, COSTbi ENROLLb, ELIGb, EBb) 

- AED (SAIb, COSTb, ENROLLb, ELlG*. EBb) 

Components and Items of Eligibility Error (>Kludes SEP/FAT 
Error) 

AED (SAIb, COSTb, ENROLLb, ELIGb, EBb) 

- AED (SAIb, COSTb, ENROLLb, ELIGb/*, EBb), 

where ELIGb/* is calculated using b values for all elements of 

eligibility error except the element(s) to be identified for error, 

which use(s) * values 



^Due to the high proportion of missing data for ENROLLb it was necessary to use an 
imputed value which was the ratio of actual disbursement to scheduled award. The 
value of scheduled award was from the institutional copy of the SAR while actual 
disbursement was obtained from the Student Record Abstract. ' 



Disbursement Error • • 

AD w AEb (SAIb, COST*, ENROLL*, ELIGb, EBb) 
' Cost of Attendance Error 

' AED (SAIb, COSTb, ENROLLb, ELIGb, EBb) 

- AED (SAIb, COST*, ENROLLb, ELIGb, EBb) 

Enrollment Status Error 

AED (SAIb, COSTb, ENROLLb, ELIGb, EBb) 

- AED (SAIb, COSTb, ENROLLb, ELIG*, EBb) 

Calculation and Accounting Error 

AD - AED (SAIb, COSTb, ENROLLb, ELIGb, EBb) 

Several features of these equations should be discussed, as they influenced the 
final equations. First, it was considered essential that the sum of overall student 
error and overall institutional error equal total error. This permits direct 
presentation of the amount of error attributable to each of these two main sources 
and maintains consistency of these key equations with Stage One. (Ii\ order to 
achieve this additivity, the * values for institutional variables were used in the 
overall student error equation instead of the b values. This has Uttle impact on the 
resulting student error because of their inclusion in both .terms of the equation.) 

Second, within the overall student error, the components and items of error 
are not additive. The overlapping contributions oi the variables to SAI prevent 
additivity, but the equation still permits their relative contributions to overall 
student error to be shown. A similar equa/ion was used in Stage Oi^, but some 
refinements have been made to assure that the relative contributions are accurately 
stated. 

/ ■ ' .. .■ . , 

Third, the two ipajor components of Institutional error — eligibility and 
(^isbursement*-are nof additive. This permits accurate presentation of the amount 
of eligibility error from a base using all actual data. 

Finally, within disbursement error, the equations for cost of attendance error, 
enrollment status error, and calculation and accounting error are not additive. This 
allows consistency in the determination of each and assures the accuracy of their 
relative contributions to error. 



